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HANDBOOK  FOR  THE  DEVELOPMENT  OF  SKILL  QUALIFICATION  TESTS 


CHAPTER  1 
INTRODUCTION 
Section  I.  GENERAL 

1-1.  PURPOSE.  The  purpose  of  this  handbook  Is  to  provide  general 
guidance  on  construction  and  validation  of  the  Skill  Qualification  Test 
(SQT)  to  personnel  responsible  for  preparing  SQT's  for  the  Active  Army, 
National  Guard,  and  Army  Reserve. 


'-2.  INTRODUCTION. 

a.  Tbe  Enlisted  Personnel  Management  System  (EPMS)  Is  designed  to 
meet  requirements  of  the  US  Army  to  Integrate  various  management  and 
training  subsystems  of  the  Army.  EPMS  features  training  and  education 
as  a progressive  career  process  dependent  upon  a comprehensive  profi- 
ciency training  and  testing  program  to  evaluate  competence  of  enlisted 
soldiers  at  all  skill  levels. 

The  proficiency  test  program  developed  to  verify  competence  Is  the 
SQT.  The  goal  of  each  SQT  Is  to  provide  an  equitable,  reliable,  and 
relevant  means  of  determining  Job  proficiency  of  enlisted  soldiers. 

b.  Five  skill  levels  under  EMPS  standardize  the  relationship  between 
a soldier's  grade  and  his  skill  level.  They  are  - 


Grade  Skill  Level  (SL) 


E-8,  E-9 

SL5 

E-7 

SLA 

E-6 

SL3 

E-5 

SLZ 

E-i  thru  E-4 


SLl 
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l-3>  TOTAL  PLAN.  The  SQT  Is  part  of  the  total  EPNS  plan  which  Includes 
development  of  an  Outline  Individual  Training  Plan  (OITP),  construction 
of  Soldier's  Manuals  and  Initiation  of  resident  and  extension  training. 


1-4.  SEQUENCE.  After  reorganization  of  career  management  fields  by 
USA  MILPERCEN,  proponent  service  schools  must  analyze  each  Job  or 
Military  Occupational  Specialty  (MOS)  to  determine  the  tasks  critical  to 
duty  performance  at  each  skill  level.  That  constrained  task  list 
becomes  the  basis  for  all  subsequent  Q1PS  actions. 

The  OITP  evolves  from  the  list  of  critical  tasks  as  each  proponent 
school  decides  on  the  training  site  for  each  task  of  a given  MOS.  The 
OITP  delineates  which  will  be  mastered  Initially  by  extension  training, 
and  which  will  be  reserved  for  initial  mastery  In  the  service  schools  or 
training  centers. 

Since  these  decisions  bear  directly  on  the  kinds  of  resources  required 
by  schools  to  provide  training  support  to  the  MOS,  the  OITP  will  be 
forwarded  to  HQ,  U.S.  Army  Training  and  Doctrine  Command,  (TRADOC)  for 
approval  not  later  than  nine  months  prior  to  the  Career  Management  Field 
(CMF)  effective  date. 

The  CMF  effective  date  Is  defined  as  the  date  upon  which  training 
will  begin  for  that  CMF  under  the  E?MS  system.  Guidance  for  construction 
of  the  Individual  Training  Plan  is  provided  In  TRADOC  Regulation  351-3. 


1-S.  SOLDIER'S  MANUAL.  Construction  of  the  Soldier's  Manual  follows 
formulation  of  the  OITP.  Draft  Soldier's  Manuals  will  be  forwarded 
to  TRADOC  for  review  and  approval  18  months  before  the  first  day  of  the 
test  quarter  for  the  first  SQT.  Subsequent  to  TRADOC  approval,  camera- 
ready  copy  of  the  Soldier's  Manual  will  be  forwarded  to  TRADOC  10  months 
prior  to  the  first  day  of  the  test  quarter  for  the  first  SQT.  This  will 
facilitate  printing  and  distribution  of  Soldier's  Manuals  to  MOS  incum- 
bents at  least  six  months  prior  to  the  SQT. 


1-6.  TESTS.  Skill  Qualification  Tests  will  be  administered  to  everyone 
In  a CMF  at  any  time  during  a test  quarter.  MOS  Incumbents  will  be 
evaluated  only  by  EER  until  an  SQT  is  available.  The  SQT  plan  for 
each  MOS  will  be  forwarded  to  HQ,  TRADOC,  18  months  before  the  first  SQT 
test  quarter.  The  final  draft  SQT  and  the  SQT  Notice  will  be  forwarded 
nine  months  before  the  first  SQT  test  quarter. 

1-7.  TASKING  LETTERS.  Tasking  letters  established  Individual  milestone 
schedules  have  been  worked  out  with  each  proponent  TDA.  Check  the 
tasking  letter  for  your  TDA  to  find  the  overall  development  plan  for 
your  SQT. 
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Section  II.  HOW  TO  USE  THIS  HANDBOOK 


1 -8.  STEPS.  This  handbook  Is  Intended  to  take  the  SQT  developer 
through  all  the  steps  to  consider  when  developing  and  validating  an  SQT. 
Sections  are  in  sequence  for  normal  SQT  development. 


1 -9.  CHECKLIST.  Most  of  the  material  in  the  handbook  is  neither  diffi- 
cult nor  revolutionary.  Developers  will  be  familiar  with  some  of  it; 
other  material  will  be  new.  To  this  end,  the  handbook  is  Intended  to 
serve  as  a self-administered  checklist  during  development. 


1-10.  DIAGRAMS.  The  material  is  presented  in  the  format  of  flow  charts 
or  decision  diagrams.  Handbook  developers  have  attempted  not  to  compli- 
cate your  Job  with  complex  diagrams;  therefore,  do  not  let  the  diagrams 
turn  you  off  without  giving  them  a try. 


Remember  a few  basic  rules  when  "reading"  the  diagrams:  an  oval(  ) 
represents  some  sort  of  input  or  starting  document  or  a final  product; 
a rectangle  [ I represents  some  type  of  required  action;  and  a diamond 
a decision,  usually  in  the  form  of  a yes/no  answer.  Other  than 
that,  simply  follow  the  flow  of  the  arrows. 


1-11.  TEXT.  Diagrams  are  expanded  on  by  text  that  explains  how  to  do 
an  action,  or  things  to  consider  when  you  make  a decision.  As  you  become 
more  familiar  with  the  process  in  each  section,  you  will  rely  less  on  the 
text  and  more  on  the  diagrams.  Even  for  experienced  test  developers, 
the  diagrams  serve  as  a checklist  to  Insure  that  all  aspects  of  SQT 
development  are  covered. 


1-12.  CAUTION.  A word  of  caution  is  needed  when  using  this  handbook. 
The  SQT  program  is  a complex  testing  system  that  touches  every  Job  and 
skill  level  that  exists  in  the  Army  Force  structure. 

No  single  document  then,  can  possibly  cover  every  question  or  situa- 
tion that  will  arise  in  the  various  Test  Development  Activities  (TDA) 
during  test  development.  So  while  the  handbook  will  serve  as  the  basis 
for  SQT  development,  there  will  be  numerous  exceptlo  <8  when  applied  to 
individual  development  problems. 

This  handbook,  then,  applies  to  "normal"  SQT  development  and  Is 
intended  as  an  aid  to  development.  The  Intent  is  not  to  have  TDA 
circumvent  the  guidance  when  it  suits  them  nor  is  it  to  hold  them 
rigidly  to  unrealistic  requirements. 
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1-1 3*  GUIDANCE.  This  handbook  also  reflects  TRADOC  guidance.  With  any 
system  as  large  and  complex  as  SQT,  further  policy  changes  cannot  be 
ruled  out.  Changes  that  affect  development  will  be  Issued  as  changes  to 
this  handbook. 
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1-14.  REFERENCES.  There  are  some  other  publications  you  should  read. 
These  are: 


a.  Letter  HQ  TRADOC  Subject:  EPMS  Implementation 
Guidance  and  Milestones  dtd  23  October  1975  w/ 
four  Incl.  (The  inclosures  give  you  the  technical 
and  printing  guidelines  necessary  for  final 
production  of  your  SQT.) 

b.  "Handbook  for  the  Administration  of  Skill 
Qualification  Tests."  Test  Edition  June  .1976. 
(Contains  guidelines  for  the  Test  Control 
Officer  for  SQT  administration*  but  includes 
descriptions  of  materials  to  be  provided 

by  the  TDA.) 

c.  "Guidance  to  Proponents:  Procedures  to  Prepare 
Manual  for  Administration"  Letter  ATTNG  TMI-ITEG, 
HQ  TRADOC,  2 April  1976. 


Section  III.  DEFINITIONS 

1-15.  COMBINATION  PRODUCT /PROCESS  TASKS.  In  HOC,  the  scorer  may  observe 
procedure  for  part  of  the  task  and  observe  outcomes  or  results  for  other 
parts  of  the  task. 

1-16.  CRITICAL  COMBAT  SYSTEMS.  The  systems  which  are  directly  Involved 
in  the  role  of  land  warfare  are  termed  combat  systems.  Combat  systems 
are  deemed  critical  when  their  operational  presence  is  pivotal  to  the 
success  of  combat.  "Systems"  is  used  in  its  broadest  sense,  in  that  a 
tank  is  a system  as  is  personnel  replacement  to  combat  units.  Examples 
of  standard  critical  combat  systems  include  all  nuclear  capable  weapons, 
intelligence  gathering  activities  and  individual  anti-armor  capabilities. 


1-17.  HANDS-ON  CCmPONENT  (HOC).  The  HOC  is  that  part  of  the  SQT  where 
the  ability  of  the  soldier  to  perform  critical  tasks  is  tested  on  actual 
job  equipment  or  simulators.  The  HOC  may  take  up  to  four  hours  to  admin- 
ister to  the  Individual  soldier.  Total  administration  time  at  a typical 
installation  will  usually  be  two  days  but  may  extend  up  to  five  days  for 
an  HOC  requiring  major  equipment  end  items  or  if  it  is  to  be  administered 
to  MOS  with  large  population.  Up  to  16  tasks  may  be  tested  in  the  HOC. 
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1-18.  INDIVIDUAL  TRAINING  EVALUATION  DIRECTORATE  (ITED).  ITED  Is  the 
activity  designated  by  the  Commander,  TRADOC,  to  develop,  Implement  and 
administer  the  SQT  system.  When  you  write  to  the  ITED,  use  the  following 
address:  Director,  Individual  Training  Evaluation  Directorate,  ATTN: 
ATTSC-IT,  Fort  Eustls,  VA  23604. 


1-19.  ITEM.  The  basic  unit  of  a written  SQT  Is  the  Item.  An  Item  con- 
sists of  written  presentation  of  Information,  or  a question  which  exam- 
inees answer  by  selecting  the  correct  response (s)  from  alternatives 
presented.  There  may  be  as  many  as  10  alternatives  per  Item. 


1-20.  JOB  EXPERT.  A Job  expert  Is  a subject  matter  expert  who  has  had 
recent  field  experience  with  soldiers  In  the  SQT  being  tested,  and  has 
supervised  the  task  In  question.  Generally  Job  experts  are  NCO's  and 
are  oftentimes  available  at  the  TDA. 

Although  the  terms  "subject  matter  expert"  and  "Job  expert"  are  often 
used  Interchangeably,  the  distinction  Is  Important  for  the  developer. 

For  example,  the  Industry  representative  for  an  Item  of  equipment  may  be 
an  SME  but  the  Individual  who  used  or  supervised  the  use  of  that  Item  In 
a unit  Is  a Job  expert. 

Job  experts  are  more  useful  to  the  developer  because  their  experience 
qualifies  them  beyond  technical  competence.  The  Job  expert  Is  always  an 
SME,  but  the  SME  Is  not  necessarily  a Job  expert. 


1-21.  KEY  ELEMENTS.  To  Insure  that  the  most  Important  part  of  the 
task  Is  tested,  key  elements  will  be  Identified.  Key  elements  are  the 
most  frequent  causes  of  failure  or  those  elements  which  have  serious 
consequences  of  failure.  Key  elements  are  Identified  by  subject  matter 
experts  with  field  experience. 


1-22.  NON-PERFORMERS.  The  opposite  of  performers.  Again  self-ratings 
are  primary  method  of  Identification,  but  other  means  also  may  be  used. 
Level  of  training  and  experience  often  can  be  used  as  blanket  predictors, 
e.g.,  soldiers  In  entry-level  training  courses  are  non-performers  on  all 
but  the  most  basic  tasks.  Non-performers  do  not  have  to  be  totally 
deficient  on  the  task  In  question.  Non-performers  are  usually  easier  to 
Identify  than  performers  but  their  role  Is  no  less  Important  In  the 
validation  process. 


1-23.  PERFORMANCE-BASED  TEST.  In  the  WC,  some  tasks  or  parts  of  tasks 
are  tested  by  asking  questions  about  how  the  task  Is  performed.  The 
questions  are  based  on  the  examinee's  knowing  how  to  perform  the  task 
correctly  even  though  he  does  not  have  to  do  It  to  answer  the  question. 
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1-24.  PERFORMANCE  CERTIFICATION  COMPONENT  (PCC).  Many  tasks  requlr*  use 
of  manipulative  or  other  motor  skills  by  the  soldier,  but  also  may  be 
expensive  tasks  In  terms  of  time,  equipment  and  other  resources.  For 
that  reason,  a PCC  may  be  Included  In  an  SQT. 

The  PCC  allows  a supervisor  to  observe  a soldier's  performance  during 
the  12  months  prior  to  the  test  quarter  and  to  rate  the  soldier's  perfor- 
mance using  the  performance  measures,  conditions,  and  standards  contained 
In  the  Soldier's  Manual. 


1-25.  PERFORMANCE  MEASURES.  After  refinement,  task  elements  become 

performance  measures  In  the  HOC.  They  are  those  steps  or  behaviors  that  ' 

the  scorer  looks  at  to  decide  If  the  soldier  Is  performing  the  task 

correctly. 


1-26.  PERFORMERS.  During  validation  of  the  components.  It  Is  necessary 
to  Identify  soldiers  who  can  be  predicted  to  successfully  complete  the 
task  being  tested.  The  primary  means  of  Identifying  performers  Is  by  the 
use  of  self-ratings,  but  may  also  Involve  supervisor  evaluating  experi- 
ence, training  and  other  Independent  measures  of  overall  proficiency. 

1-27.  PROCESS  TASKS.  In  the  HOC,  some  tasks  are  scored  by  observing 
the  process  the  soldier  goes  through  In  task  performance.  Each  step  or 
element  Is  scored  as  It  Is  performed  In  a process  task. 

1-28.  PRODUCT  TASKS.  The  opposite  of  process  tasks.  By  looking  t a 
result  oi  outcome,  the  scorer  can  Infer  whether  the  task  was  performed 
correctly. 

1-29.  SCORABLE  UNIT  (SU).  The  term  used  to  describe  the  means  of 
testing  a task  for  the  WC,  HOC,  and  PCC.  In  the  HOC  and  PCC,  a SU 
consists  of  up  to  20  performance  measures.  In  the  WC  the  SU  can  consist 
of  up  to  10  Items. 

When  scoring  the  SQT,  one  point  Is  given  for  each  correct  SU.  For 
normal  SQT  development,  a single  SU  Is  used  to  measure  each  task,  but 
you  may  have  more  SU  If  the  task  Is  especially  complex  and  Important. 

An  SQT  can  contain  between  30  and  76  SU. 


1-30.  SKILL  QUALIFICATION  TEST  (SQT).  The  SQT  Is  a validated  perfor- 
mance-oriented test  of  the  soldier's  ability  to  perform  the  critical 
tasks  required  by  his  Army  job.  The  SQT  Is  used  to  assess  Job  competence 
for  training  feedback  and  personnel  management  purposes.  Each  SQT  may 
have  up  to  three  components. 


J 


ARI  P-77-5 

1-31.  SOLDIER'S  MANUAL  (SM) . The  SM  is  a book  that  lists  for  the  soldier 
critical  tasks  needed  to  perforin  satisfactorily  at  his  present  skill  level 
and  the  next  higher  skill  level  within  his  MOS.  In  addition,  It  tells  the 
soldier  how  to  perform  the  tasks,  the  expected  conditions  under  which  they 
will  be  performed,  and  the  standards  which  must  be  met.  The  SM  Is  the 
basis  for  the  tasks  used  In  the  SQT. 


1-32.  SUBJECT  MATTER  EXPERT  (SME).  The  SQT  developer  will  be  preparing 
tests  for  a number  and  variety  of  tasks.  The  developers'  depth  of  knowl- 
edge of  these  tasks  will  vary.  During  development  It  Is  necessary  to  make 
use  of  Individuals  who  have  a thorough  working  knowledge  of  the  task  to 
assist  in  review  of  task  analysis  and  answer  questions  during  development. 
These  individuals  are  generally  available  within  the  TDA,  either  as  co- 
workers or  within  other  departments. 


1-33.  SQT  NOTICE.  Many  tasks  cannot  be  tested  In  the  SQT  the  same  way 
they  are  detailed  In  the  SM.  The  purpose  of  the  SQT  Notice  Is  to  tell 
the  soldier  which  casks  will  be  In  the  SQT,  and  to  explain  Co  him  how 
they  will  be  tested.  The  SQT  Notice  is  given  to  the  soldier  60  days 
before  the  test  quarter,  and  serves  as  the  basis  for  training  on  those 
tasks  contained  In  the  SQT. 


1-34.  TASK.  For  SQT  development  only,  a task  Is  an  action  performed  by 
a single  Individual  which  produces  a result  or  product  that  can  be  mea- 
sured (scored).  A Cask  has  an  Identifiable  start  point  and  an  identifi- 
able completion  point  and  Intermediate  actions  that  can  be  specifically 
stated  In  terms  of  behavioral  activities. 

(CAUTION;  There  are  many  different  ways  to  define  a task.  This 
definition  Is  not  meant  Co  be  all  encompassing  or  to  serve  other  purposes 
which  require  definition  of  a Cask.  It  Is  Intended  only  as  a working 
definition  for  purposes  of  SQT  development.) 


1-35.  TASK  ANALYSIS.  Task  analysis  Is  the  written  description  of  how  a 
task  Is  performed.  It  Includes  the  following:  condltlQns  of  performance, 
equlpme-.t  and  tools  required,  actions  performed  (broken  out  Into  steps  or 
activities)  and  the  standard  for  final  products,  results  or  outcomes. 

The  task  analysis  Is  the  basis  for  all  developmental  work  In  the  SQT. 

1-36.  TASK  ELEMENTS.  The  behaviors  or  steps  in  a task  comprise  the  ele- 
ments of  the  task.  Elements  are  taken  initially  from  the  task  analysis. 

It  is  sometimes  necessary  Co  add  task  elements  prior  to  test  development. 
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1-37.  TEST  CONTROL  OFFICER  (TCO).  The  Individual  responsible  for  the 
administrative  functioning  of  SQT  in  the  field.  He  is  the  point  of 
contact  between  the  field  and  ITED.  He  may  be  assigned  within  the 
operations  staff  section  or  an  administrative  staff  section. 


1-38.  TEST  DEVELOPMENT  ACTIVITY  (TDA).  The  TDA  are  schools  and  other 
organizations  that  develop  and  validate  SQT  and  Interact  with  the  ITED 
in  their  review.  There  are’ approximately  34  TDA  in  the  SQT  program. 


1-39.  TRACKED  TEST.  Some  duties  and  tasks  within  MOS  may  vary  greatly 
in  the  way  they  are  performed.  This  may  be  due  either  to  differences  in 
equipment,  e.g.,  the  M60A1  tank  vs.  M60A2,  or  because  the  Jobs  (duty 
positions)  at  the  same  skill  level  in  on  MOS  are  actually  different, 
e.g..  Mortar  Crewman  vs.  Forward  Observer  in  Skill  Level  1 of  MOS  IIC. 

It  would  be  unfair  to  test  all  incumbents  within  the  MOS  the  same  way. 

For  this  reason,  some  SQT  have  portions  of  the  test  which  are  tailored 
to  a specific  duty  position  or  type  of  equipment.  These  are  called 
tracked  teats  and  are  identified  as  such  in  the  SQT  Notice. 

1-40.  VALIDATION.  An  Important  aspect  of  the  SQT  is  that  all  scorable 
units  are  pre-tested  to  assure  that  they  discriminate  between  performers 
and  non-performers  and  that  they  are  perceived  as  fair  by  a representa- 
tive sample  of  soldiers  who  will  take  the  test.  Validation  also  confirms 
that  the  items  can  be  understood  by  the  soldier  and  that  HOC  items  are 
scored  the  same  by  different  scorers. 


1-41.  WRITTEN  CCmPONENT  (WC).  The  NC  tests  30-50  tasks  (scorable  units). 
Each  scorable  unit  is  comprised  of  up  to  10  questions,  with  each  question 
having  2 to  10  possible  responses.  The  WC  is  used  to  test  those  tasks 
requiring  decisionmaking  or  other  mental  skills  on  the  part  of  the 
soldier. 


1-42.  WRITTEN  PERFORMANCE  TEST.  In  the  WC,  some  tasks  or  parts  of  tasks 
can  be  performed  almost  exactly  like  they  are  on  the  Job.  The  tasks  are 
characterized  by  the  fact  that  the  correct  answer  cannot  be  recognized 
without  going  through  the  task  procedure. 
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Section  IV.  SQT  DEVELOPMENT  PROCESS  OVERVIEW 

I 

1-43.  INTRODUCTION.  The  SQT  development  process  Includes  constructing  I 

and  validating  each  component.  The  process  requires  close  coordination 

between  the  TDA  and  ITED.  The  development  process  Is  also  d)mamlc 

because  there  are  many  opportunities  to  revise  scorable  units  before  the 

SQT  Is  final. 


1-44.  SOLDIER'S  MANUAL  TASK  LIST.  The  starting  point  for  SQT  develop- 
ment Is  the  current  or  draft  Soldier's  Manual.  Just  as  the  Soldier's 
Manual  tells  the  soldier  what  he  must  do.  It  tells  you  what  group  of 
tasks  to  test. 


1-45.  SELECT  TASKS  FOR  TESTING.  Most  MOS  have  so  many  tasks  that  It  Is 
not  practical  to  test  all  of  them  In  an  SQT.  But  selection  of  tasks  Is 
not  random  or  haphazard.  Tasks  to  test  are  determined  through  a system- 
atic evaluation  of  field  data  and  Input  from  the  senior  officer  respon- 
sible for  the  MOS. 


1-46.  REVIEW  TASK  ANALYSIS.  Probably  the  single  most  Important  aspect 
of  the  SQT  development  process  Is  review  of  the  task  analysis.  The 
effort  during  the  review  process  will  determine  adequacy  of  the  final 
test . 


1-47.  ALLOCATE  TASKS  TO  COMPONENTS.  The  SQT  allows  the  developer  three 
options  for  testing  - the  Written  Component  (WC),  Hands-On  Component 
(HOC),  and  Performance  Certification  Component  (PCC).  Currently  the  only 
requirement  Is  that  each  test  must  have  a WC.  But  there  are  desirable 
options  of  having  an  HOC  and  a PCC.  Consider  the  characteristics  of  the 
task  when  you  decide  the  best  component  to  use  for  testing  each  task. 


1-48.  PREPARE  SQT  PLAN. 


a.  You  will  prepare  a comprehensive  plan  for  each  test  you  develop. 
The  SQT  Plan  Is  submitted  In  two  parts.  One  Is  a list  of  tasks  to  be 
tested  by  component;  the  other  Is  an  explanation  of  the  rationale  used 
In  selecting  and  allocating  the  tasks. 


b.  The  SQT  Plan  must  be  submitted  to  ITED  for  approval.  ITED 
will  reply  to  the  plan  within  two  working  days.  If  the  plan  Is  deficient, 
ITED  will  respond  with  specific  questions.  Once  final  approval  Is 
obtained  from  ITED,  the  plan  becomes  the  guide  for  all  developmental 
work  In  the  SQT  and  the  plan  cannot  be  changed  without  ITED  approval. 
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1-49.  DEVELOP  SQT  SAMPLE. 

a.  When  the  SQT  Plan  has  received  local  (TDA)  approval,  work  can 
begin  on  the  SQT  sample.  The  following  Items  must  be  submitted  to  ITED 
as  part  of  the  sample: 

(1)  Five  written  scorable  units. 

(2)  Two  hands-on  scorable  units  (if  used). 

(3)  All  performance  certification  scorable  units 
(If  used). 

(4)  The  SQT  Notice  (less  the  component  lead- 
in  paragraphs)  for  the  scorable  units 
submitted  In  the  sample. 

(5)  The  task  analysis  for  the  scorable  units 
submitted  in  the  sample  with  key  elements 
Identified. 


b.  The  scorable  units  must  be  submitted  as  complete  as  they  will 
be  for  testing.  ITED  will  reply  to  the  sample  within  10  working  days 
after  receipt.  If  the  sample  Is  approved,  work  may  begin  on  development 
of  the  complete  test.  If  the  sample  Is  not  acceptable,  ITED  will  return 
specific  guidance  for  corrections.  You  must  make  the  required  changes 
and  submit  a second  sample  for  any  deficient  component. 


1-50.  CONSTRUCT  SQT.  As  soon  as  ITED  approves  the  sample,  begin  devel- 
oping the  rest  of  the  SQT. 

1-51.  TRY  OUT  SQT  LOCALLY.  You  must  try  out  each  hands-on  and  written 
scorable  unit  locally  before  you  submit  the  proposed  SQT  to  ITED.  This 
tryout  Is  the  first  step  In  validation.  Check  out  each  scorable  unit 
for  completeness,  job  relevance,  doctrinal  accuracy  and  perceived  fair- 
ness. Also  check  the  clarity  and  readability  of  WC  Items  and  the  relia- 
bility of  scoring  for  the  HOC. 


1-52.  VALIDATE  SQT  IN  FIELD. 

a.  The  field  validation  for  the  HOC  has  two  phases.  The  first 
phane  Is  to  check  the  reliability  of  scoring  each  scorable  unit  when 
representative  soldiers  are  tested.  When  this  Is  done,  the  TDA,  in 
conjunction  with  ITED,  selects  a field  unit  for  tryout  of  the  complete 
HOC. 
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CHAPTER  2 

SELECT  TASKS  FX)R  TESTING 


'I 

b 
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, 2-1.  INTRODUCTION. 

a.  Each  skill  level  of  ai^  MOS  is  defined  by  a list  of  tasks  that  a 
soldier  Is  expected  to  be  able  to  perform.  The  comprehensive  task  list 
Is  contained  In  the  appropriate  Soldier's  Manual.  The  SQT  will  Include 
tasks  from  a soldier's  present  and  next  higher  skill  levels. 

As  a general  rule  for  skill  levels  one  through  four,  tvo-thlrds  to 
three-fourths  of  the  tasks  In  the  SQT  will  measure  the  soldier's  compe- 
tence at  his  present  skill  level,  while  the  remainder  of  the  one-fourth 
to  one-third  of  the  tasks  will  measure  his  competence  at  the  next  higher 
skill  level. 

For  the  higher  skill  levels  or  when  MOS  merge,  the  mixture  may  be  up 
to  50/50,  and  In  some  MOS  there  may  be  little  or  no  distinction  among 
tasks  between  skill  levels. 

b.  The  test  developer  Is  faced  with  the  problem  of  selecting  a 

I small  number  of  tasks  from  all  the  tasks  In  the  soldier's  Job.  The  SQT 

Is  not  necessarily  Intended  to  be  a random  sample  of  the  different  kinds 
of  tasks  that  a soldier  Is  expected  to  be  able  to  perform. 

I This  distinction  Is  Important.  Tasks  are  not  selected  for  testing 

merely  because  they  are  representative  or  because  most  soldiers  In  the 
> MOS  must  be  able  to  perform  them.  Testing  resources  are  a precious 

\ commodity  and  tasks  should  be  selected  that  will  provide  a high  "payoff 

I for  the  effort  necessary. 

To  a large  extent,  SQT  testing  will  Influence  training  so  the  test 
I developer  has  a further  responsibility  In  his  task  selection.  This 

section  Is  concerned  with  the  procedure  for  selecting  "best"  tasks  for 
Inclusion  In  the  SQT. 

c.  Before  discussing  methodology  for  task  selection.  It  Is  impor- 
tant to  emphasize  the  role  of  the  senior  officer  In  task  selection.  The 
senior  officer  may  be  the  Assistant  Commandant  (AC)  or  the  Director  of 
Training  Development  (DOTD)  at  the  TDA. 

In  either  case.  It  Is  Important  to  obtain  the  senior  officer's 
guidance  and  concurrence  In  task  selection.  You,  as  the  test  developer, 
must  Insure  that  the  chain  of  command  Is  exercised  early  In  the  task 
development  process  to  Insure  concurrence  with  the  method  of  task 
* selection  and  with  the  tasks  selected. 
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The  senior  officer  is  a valuable  resource  for  identifying  Casks  for 
testing  because  he  has  the  broadest  perspective.  The  procedure  will 
certainly  vary  at  different  TDA.  The  requirement  to  get  the  senior 
officer  Involved  in  task  selection  does  not  imply  that  each  discrete 
task  must  have  General  Officer  or  0-6  approval. 

Categories  of  tasks  and  Che  method  of  selection  must  reflect  the 
senior  officer's  input  before  development  is  started. 

d.  No  single  method  of  task  selection  will  meet  the  requirements 
of  all  TDA.  We  will  discuss  them  here  without  regard  Co  priority, 
although  some  methods  are  more  desirable.  The  TDA  must  select  the  method 
or  combination  of  methods  Chat  will  provide  them  with  the  best  sample  of 
Casks  based  on  information  available.  Most  TDA  will  use  more  Chan  one 
method.  The  important  thing  to  ask  is,  "Will  this  task  selection  method 
provide  the  highest  payoff  for  Che  resources  I'm  expending  on  this  test?" 

e.  The  methods  are: 

• Known  performance  deficiencies. 

• Critical  combat  systems. 

• ARTEP's. 

• Evaluation  results. 

• Job  content.  i| . 


2-2.  KNOWN  PERFORMANCE  DEFICIENCIES.  The  purpose  of  this  method  is  to 
select  tasks  that  tend  to  be  performed  poorly  by  most  people  in  the  MOS. 

If  only  one  method  of  task  selection  could  be  used,  it  would  likely  be 
this  one.  The  difficulty  with  its  use  is  in  obtaining  sufficient  infor- 
mation on  which  to  base  judgments.  Below  are  discussed  some  more 
generalized  sources  that  should  be  considered. 

a.  TOE  Manpower  Authorization  Criteria  (MACRIT).  DARCOM  maintains 
a master  data  file  of  annual  direct  productive  maintenance  man-hours 
(AMMH)  for  all  standard  and  limited  procurement  items  of  equipment  (as 
listed  in  AMC  Supplement  One  to  AR  570-2,  Manpower  and  Equipment  Control). 

This  file  is  maintained  at  USAMMC  and  the  AMMH  data  is  updated 
annually  by  the  commodity  commands.  The  AMMH  data  is  furnished  to  TRADOC 
requesting  activities.  The  AMMH  printout  includes  operator/crew  mainte- 
nance requirements  codes,  supporting  maintenance  MOS  and  Organizational, 

DS  and  GS  AMMH  breakouts. 

b.  Red  Team  Assessment  Programs.  The  Red  Team  Assessment  program 

is  designed  to  provide  CG,  DARCOM  with  an  assessment  of  Selected  Critical 
Field  Systems.  The  appendixes  to  this  circular  list  the  systems  and 
the  schedule  of  assessment  reporting. 
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performance  deficiencie: 


List  tMki  supporting 
system 
(per.  2-3) 


List  tasks  contributing 
to  ARTEP 
(par.  2-4) 


List  tasks  relating 
to  performance  data 
(par.  2-5) 


Job  content  sampie 
(par.  2 6) 


Figure  2-1 . Select  Tasks  for  Testing 
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c.  U.S.  Army  Maintenance  Management  Center  (AMMC)  (Bluegrass  Army 
Depot,  Lexington,  KY  40511)  collects  and  maintains  a variety  of  equip- 
ment-maintenance related  reports  and  data.  These  Include: 

(1)  DA  Sample  Data  Collection  (SDC).  AR  750-37  specifies  a proce- 
dure for  collection  of  maintenance  data  utilizing  a sampling  technique. 
The  U.S.  Army  Maintenance  Management  collects  and  analyzes  data  on 
selected  Items  of  equipment  or  to  study  specific  problems. 

(2)  Reports  furnished  by  comodlty  commands  to  AMMC  for  Command 
specific  Items  and  problems.  These  reports  Identify  equipment  with 
high  degrees  of  fault  and  deficiencies. 

d.  Technical  Proficiency  Inspections  (TPI)  and  Technical  Standardi- 
zation Inspection  (TSI).  These  results  are  available  from  major  Head- 
quarters such  as  HQ  USAREUR  and  7th  Army  and  FORCE  command. 

e.  IG  Inspection  results  are  also  available  from  Major  Command  Head- 
quarters. They  should  be  requested  by  specific  topic. 

f.  Direct  Mall  Questionnaires  (DMQ). 

(1)  For  some  TDA,  the  only  method  of  Identifying  specific  deficien- 
cies may  be  through  the  use  of  a direct  mall  questionnaire  to  field 
commanders.  This  may  be  used  also  In  conjuctlon  with  any  of  the  sources 
above.  The  level  of  command  to  obtain  the  Information  from  Is  best 
determined  by  the  TDA. 

Too  low  a level  may  reflect  only  a local  view  (plus  Increase 
the  number  of  responses  which  must  be  tabulated)  while  too  high  a level 
In  the  command  structure  may  not  accurately  reflect  the  specific 
deficiencies. 


The  format  of  the  questionnaire  can  be  tailored  to  fit  the 
type  of  Information  desired  by  the  TDA,  but  the  Information  requested 
should  be  as  specific  as  possible.  Do  not  ask  the  commander  for  a 
subjective  analysis  of  what  he  thinks  should  be  tested. 

(2)  CAUTION:  Before  using  the  DMQ,  you  must  coordinate  with 
MILPERCEN.  Data  banks  already  exist  on  many  of  the  CMF  and  the 
Information  you  seek  may  already  be  available.  MILPERCEN  must 
also  approve  questionnaires  sent  to  the  field.  For  MILPERCEN 
guidance  on  use  of  mall  questionnaires  coordinate  through  ITED. 
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g.  It  is  recognized  that  most  of  the  above  sources  (except  the  DMQ) 
are  hardware  or  equipment  oriented  and  may  not  be  equally  useful  to  all 
TDA.  But  each  TDA  should  evaluate  information  available  from  these 
sources  before  discarding  them*  You  must  also  be  aware  of  the  time 
requirements  involved  in  using  these  sources* 

Although  the  sources  are  responsive  in  providing  available  data, 
it  must  still  be  analyzed*  A test  developer  who  faces  a two-week 
deadline  to  ITED  is  probably  not  in  a position  to  access  and  analyze  the 
information*  TAP  THE  RESOURCES  EARLY,  preferably  by  maintaining  updated 
source  documents* 


2-3*  CRITICAl.  COMBAT  SYSTEMS* 

a*  Tasks  chosen  for  testing  may  be  selected  on  the  basis  of  their 
contributions  to  the  successful  operation  oi^  maintenance  of  critical 
combat  systems*  Critical  combat  systems  are  equipment  items  or  other 
operations  that  are  pivotal  to  a unit's  success  in  combat* 

The  critical  combat  system  will  vary  from  one  MOS  to  another  as  will 
its  usefulness  as  a task  selection  tool*  Some  TDA  will  tend  to  feel  that 
almost  all  of  their  tasks  are  related  to  critical  combat  systems,  while 
other  TDA  may  find  their  links  to  critical  combat  systems  tenuous  at  best* 

For  either,  the  critical  combat  systems  approach  will  be  of  limited 
use*  The  usefulness  of  the  approach  occurs  primarily  in  the  area  of 
combat  service  support  and  to  a lesser  extent  in  combat  support  where  a 
distinction  can  be  made  regarding  activities  relating  directly  to 
critical  combat  systems* 

For  example,  the  tasks  that  are  directly  involved  in  divisional 
support  of  maneuver  units  would  be  more  suitable  for  selection  than 
tasks  which  are  not*  An  example  of  the  critical  combat  system  approach 
occurs  in  the  track  vehicle  mechanic  MOS  supporting  an  artillery  unit* 

The  tracked  carrier  for  an  artillery  piece  having  nuclear  capability 
is  a critical  combat  system*  Tasks  directly  related  to  repair  of  that 
vehicle  would  be  more  important  for  testing  than  tasks  required  to  repair 
a track  vehicle  without  a nuclear  capable  piece* 

b*  To  use  the  approach,  first  identify  the  critical  combat  systems 
that  are  related  to  the  MOS,  then  list,  from  the  Soldier's  Manual, 
individual  tasks  that  relate  directly  to  that  system* 
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2-4.  ARTEP. 

a.  The  Army  Training  Evaluation  Program  (ARTEP)  should  be  examined 
when  considering  tasks  for  SQT  development.  The  ARTEP,  of  course, 
describes  unit  tasks  while  the  SQT  Is  based  on  Individual  tasks. 
Therefore,  you  must  Identify  the  individual  tasks  comprising  unit  tasks 
or  missions  in  the  ARTEP. 

b.  Presently,  consolidated  ARTEP  results  are  not  available,  how- 
ever Individual  unit  ARTEP  experience  may  be  gathered  through  the  DMQ 
described  above.  Validation  results  of  ARTEP  are  available,  however 
(usually  In  TDA  Directorate  of  Evaluation)  and  these  should  be  reviewed. 
Here  again,  look  for  deficiencies  In  crew-unit  performance  that  can  be 
translated  Into  Individual  tasks  for  use  In  the  SQT. 


2-5.  DIRECTORATE  OF  EVALUATION  (DOE).  Most  TDA  already  have  estab- 
lished a Directorate  of  Evaluation  and  this  organization  has  the  respon- 
sibility for  varied  evaluation  activities  proponent  to  the  TDA.  You 
must  be  familiar  with  the  Information  gathered  by  the  DOE  and  match  the 
Information  with  the  SQT  development 

Besides  the  above  mentioned  ARTEP  validations  results,  the  DOE  will 
most  likely  be  the  repository  of  results  from  SQT  already  administered. 


2-6.  JOB  CONTENT.  Just  as  known  performance  deficiencies  are  the 
most  desirable  tool  for  task  selection,  sampling  on  the  basis  of  lob 
content  Is  probably  least  desirable.  But  It  may  occur  within  a TDA  that 
no  alternative  exists  or  that  even  If  above  sources  are  used,  too  few 
tasks  have  been  selected.  If  the  expertise  and  experience  available  to 
you  Is  Insufficient  to  enable  you  to  make  Judgments  as  to  degrees  of 
criticality  for  testing,  you  will  probably  need  to  resort  to  sampling 
from  the  Soldier's  Manual  to  "fill  out"  the  SQT.  However,  even  here 
the  sampling  Is  not  done  on  a totally  random  basis.  Tasks  should  be 
organized  into  content  or  functional  areas,  e.g.,  duty  categories,  and 
a proportional  sample  drawn  from  each.  This  provides  a combined  sample 
representative  of  the  full  range  of  tasks  In  the  Soldier's  Manual. 

The  resulting  sample  of  tasks  should  be  approved  by  senior  officers  as 
discussed  before. 


2-7.  GROUP  TASKS.  If  you  used  more  than  one  method  or  source  for  task 
selection,  you  may  have  an  unorganized  list  of  tasks  for  scorable  unit 
development  at  this  point.  If  so,  sort  the  tasks  In  order  of  priority 
for  testing.  This  need  not  be  a precise  rank  ordering. 
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Tasks  may  be  roughly  grouped,  e*g.,  top,  middle,  bottom.  In  terms  of 
priority  for  testing.  The  order  of  this  list  Is  not  final  and  no  tasks 
are  being  discarded  at  this  point.  Ordering  of  tasks  Is  only  a tool  to 
use  In  planning  allocation  of  development  efforts. 

The  purpose  of  this  grouping  Is  two-fold.  First,  you  will  use  It 
when  allocating  tasks  for  testing  In  Chapter  IV.  Just  as  important;  you 
may  have  Identified  tasks  that  you  will  want  to  test  regardless  of  subse- 
quent testing  restrictions  that  you  may  encounter.  For  example,  your  AC 
may  have  specified  tasks  that  must  be  tested. 

You  certainly  want  to  Identify  the  tasks  In  some  manner  as  having  the 
highest  priority  for  testing.  You  may  have  other  tasks  that  were  Identi- 
fied by  less  reliable  criteria  which  you  may  consider  dropping  later 
from  the  test  If  problems  develop  with  size  or  feasibility. 

In  ordering  the  tasks,  developers  may  use  differing  rationale, 
but  base  your  list  on  the  Information  received  from  the  sources  listed 
above.  The  larger  the  list  of  tasks  from  these  sources,  the  more 
Important  the  ordering  process. 


2-  8.  HAVE  ENOUGH  TASKS  BEEN  SELECTED?  At  this  point  you  should  have  a 
rough  Idea  of  how  tasks  will  logically  fit  together  Into  scorable  units. 
You  should  have  a pool  of  about  30  tasks  at  this  point,  even  though  you 
have  not  decided  which  component  is  best  for  testing  any  of  the  tasks. 

If  you  do  not  have  enough  tasks,  add  tasks,  always  using  data  available 
from  one  or  more  of  the  task  selection  methods. 


2-  9.  ORDERING.  You  should  have  now  a roughly  ordered  list  of  tasks 
that.  In  addition  to  the  basic  data  sources,  reflect  guidance  from 
senior  officers.  Now,  a final  review  by  the  AC  or  Director  of  Training 
Development  should  be  obtained  for  concurrence  prior  to  actual  test 
development. 


2-10.  TASK  SELECTION.  Although  the  need  to  start  the  task  selection 
process  early  has  been  stated  before.  It  bears  reemphasls.  In  the  past, 
since  some  developers  have  been  faced  with  the  problem  of  short  lead 
time  to  meet  suspense  dates  for  SQT  submission,  task  selection  has  been 
based  on  a limited  and  subjective  analysis. 

To  Insure  that  task  selection  is  based  on  the  best  Information  avail- 
able, the  selection  process  must  be  started  well  In  advance  of  the 
development  phase.  It  Is  much  easier  to  make  adjustments  with  later 
information  than  to  force  a hurried  and  inadequate  task  selection. 
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CHAPTER  3 


REVIEW  TASK  ANALYSIS 


3-1.  INTRODUCTION. 

a.  Importance  of  a good  task  analysis  In  SQT  development  cannot  be 
overstated.  Experience  has  shown  that  when  test  developers  have  problems 
In  later  stages  of  development,  che  trouble  can  be  traced  directly  to 
deficiencies  In  the  task  analysis. 

A thorough  task  analysis  Is  essentlfl  to  the  development  of  a good 
scorable  unit,  that  Is,  a scorable  unit  relevant  to  measurement  of  task 
performance.  What  the  developer  does  between  task  selection  and  final 
validation  Is  based  on  presumed  existence  of  a good  task  analysis. 

A "bad"  task  analysis  will  result  In  an  Invalid  test  - a test  that 
simply  does  not  measure  what  the  task  requires.  No  matter  how  well  you 
do  your  Job,  If  the  work  Is  based  on  an  Inadequate  task  analysis,  the 
test  will  not  be  good. 

b.  Task  analysis  probably  still  does  not  follow  a standardized 
format  among  TDA  and  may  be  referred  to  as  a task  data  card,  training 
analysis  worksheet,  task  detailing  or  task  structure  analysis. 

The  name  or  format  Is  unimportant  for  the  developer  although  It  must 
be  uniform  within  the  TDA.  TDA  will  also  vary  In  terms  of  how  active  a 
role  SQT  developers  play  In  task  analysis. 

However,  It  Is  Important  that  SQT  developers  have  Input  Into  the  task 
analysis  program.  Task  analysis  serves  many  purposes,  one  of  the  most 
Important  being  SQT  development.  Therefore,  you,  the  developer  should 
make  known  your  needs  to  whoever  Is  responsible  for  task  analysis. 

During  review  of  task  analyses,  gaps  and  errors  are  often  discovered. 
It  has  been  the  experience  of  some  TDA  that  once  a task  analysis  has  been 
approved.  It  Is  extremely  difficult  to  change.  This  tendency  must  be 
overcome.  The  task  analysis  Is  the  building  block  from  which  both  the 
SQT  and  the  Soldier's  Manual  are  constructed;  all  three  must  track  In 
the  final  review. 

c.  It  Is  not  the  Intent  here  to  teach  you  how  to  do  a task  analysis 
(that  would  be  another  complete  area  of  study)'.  Nor  is  It  Intended  to 
criticize  existing  or  ongoing  task  analysis  efforts. 

Rather  It  Is  Intended  to  communicate  how  to  review  a task  analysis 
to  Insure  It  contains  Information  needed  for  test  development.  It  is 
important  that  you  follow  a logical  review  process  In  order  to  Identify, 
as  early  as  possible.  Information  gaps  In  existing  task  analysis  data. 
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When  deficiencies  are  discovered,  you  most  likely  will  have  to  use 
your  own  resources  to  gather  required  Information*  But  you  must  also 
Insure  that  you  coordinate  with  the  people  responsible  for  task  analysis 
to  Insure  they  are  aware  of  your  problems  and  requirements* 

3-2*  ARE  TASK  ELEMENTS  THE  SAME  IF  CONDITIONS  VARY? 

a*  Task  elements  are  steps  or  procedures  In  performing  the  task* 
Conditions  are  factors  which  Influence  performance  of  the  task*  Most 
conditions  are  associated  with  environment,  that  Is,  weather,  temperature, 
light,  precipitation  and  the  like*  Conditions  may  also  Include  equip- 
ment, terrain,  mission  or  location* 

b*  The  task  analysis  generally  will  contain  a statement  of  condi- 
tions under  which  the  task  is  performed*  Some  tasks  may  change  In  how 
they  are  performed  If  conditions  are  changed*  Therefore  the  developers 
must  consider  the  task  under  all  probable  conditions* 

Sometimes  the  "changed"  task  will  become  a new,  separate  task  state- 
ment* Changing  conditions  sometimes  alter  steps  or  elements  In  task 
performance,  and  sometimes  simply  make  the  task  more  or  less  difficult* 

For  example,  disassembly  of  the  M16  rifle  In  the  dark  under  freezing 
rain.  Is  certainly  more  difficult  than  Indoors  at  a well-lighted  work- 
bench* But  the  elements  In  performing  the  task  remain  the  same*  Other 
tasks,  however,  will  be  changed  by  conditions,  e*g*,  vehicle  maintenance 
after  fording  operations* 

3-3*  IS  THE  TASK  CRITICAL  UNDER  VARIOUS  CONDITIONS? 

a*  If  you  decide  that  a task  has  different  elements  under  different 
conditions  (and  Is  not  Included  In  the  existing  Soldier's  Manual  or  task 
analysln),  you  may  have  to  decide  If  the  "new"  task  Is  critical  for 
testing* 

But  before  that  decision  can  be  made,  those  responsible  for  the 
Soldier's  Manual  must  first  decide  If  the  task  warrants  inclusion  In  the 
manual*  If  the  task  Is  not  Judged  to  be  Job-critical,  and  therefore  not 
In  the  SM,  It  does  not  become  a candidate  for  testing* 

On  the  other  hand.  If  task  analysis  personnel  decide  the  new  task 
belongs  In  the  SM,  you  must  then  decide  whether  to  Include  the  task  In 
the  SQT*  Follow  the  procedures  outlined  In  Chapter  2 when  you  make  that 
decision* 
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b.  If  elements  of  the  task  remain  the  same  under  various  conditions, 
there  is  no  requirement  to  obtain  more  task  analysis  data.  If  the 
elements  do  change,  treat  the  task  like  a new  task. 

(1)  Is  the  new  task  critical?  If  the  answer  is  "no," 
discard  the  task. 

(2)  If  the  answer  is  "yes,"  obtain  task  analysis  data 
and  review  it. 


3-4.  DOES  TASK  STATEMENT  DEFINE  A TASK? 

a.  Definition  of  a task  is  imprecise  and  although  a quantity  of 
literature  exists  on  defining  a task,  again  it  is  intended  not  to 
involve  the  test  developer  in  an  area  that  belongs  to  task  analysis 
personnel. 

However,  during  review  of  the  task  analysis,  insure  that  the  task 
statement  describes  what  is  to  be  performed.  The  task  statement  must 
define  an  action  and  the  task  statement  must  have  ^ measurable  outcome. 

For  purposes  of  SQT,  knowledges  are  not  considered  tasks;  the  essence 
of  a Job  task  is  doing  something,  not  knowing  something.  Compare  the 
task  elements  as  listed  with  the  task  statement  and  Insure  that  the  state- 
ment is  the  activity  described. 

Be  cautious  about  tasks  that  are  actually  sub-tasks.  Some  sub-tasks 
may  be  tested  as' tasks,  depending  on  the  skill  level  of  the  MOS  I'nvolved. 
But  normally  the  end  task  should  be  tested,  not  a series  of  sub-tasks 
tested  separately. 

b.  Be  careful  of  generalized  statements  expressing  attitudes, 
methods  or  non-concrete  behaviors.  Generalizations  often  arise  when 
attempting  to  describe  duties  of  NCO's.  Be  alert  for  verbs  such  as 
"Inspect,"  "supervise,"  "direct,"  "counsel." 

It  will  be  necessary  to  obtain  more  task  analysis  information  as  to 
exactly  what  these  incumbents  do.  If  the  actions  are  not  identified  at 
the  start,  you  may  well  expend  much  time  and  effort  only  to  find  you  have 
a "task"  that  cannot  be  tested  as  written. 

c.  If  the  task  statement  does  not  describe  a task  or  does  not  corres- 
pond to  the  elements,  the  statement  must  be  rernrltten.  This  is  Importent 
for  two  reasons.  First,  you  must  know  what  activity  you  are  going  to 
test  and  second,  the  soldier  who  will  be  tested  must  be  able  to  identify 
easily  what  performance  he  is  expected  to  accomplish. 


I 


•v 


'I 


ARI  P-77-5  f 


d«  You  may  find  a task  statement  too  broad  for  what  should  be  tested 
at  a certain  skill  level*  Yet  because  the  Soldier's  Manual  has  been 
distributed,  you  find  the  task  analysis  Is  "locked-ln."  If  the  broad 
task  Is  actually  an  aggregate  of  several  subtasks,  you  should  treat  the 
subtasks  separately  for  test  purposes  and  use  the  SQT  Notice  to  Inform 
the  soldier  of  additional  "tasks"  to  be  tested. 


3-5.  ARE  TASK  LIMITS  APPROPRIATE  TO  THE  SKILL  LEVEL? 

a.  This  decision  Is  similar  to  the  preceding  decision  but  focuses  on 
a more  precise  aspect  of  soldiers'  performance.  Task  limits  are  param- 
eters of  the  job  as  they  apply  to  soldiers  at  the  skill  level  considered. 
The  developer  Is  deciding  where  the  task,  as  performed  by  the  soldier, 
begins  and  ends  and  %fhat  activity  It  encompasses. 

b.  Task  limits  means  different  activities  for  different  levels  of 
soldiers.  A first  sergeant  may  prepare  a draft  reply  to  a congressional 
Inquiry  and  his  clerk  may  type  It.  Yet  the  task  "Prepare  Correspondence" 
Is  often  listed  for  both  duty  positions. 

c.  Problems  %rlth  task  limits  are  caused  generally  because  the  task 
statement  Is  too  broad.  Restate  the  task  In  terms  of  what  the  Incumbent 
performs,  then  check  to  Insure  that  task  elements  accurately  describe 
that  performance. 

3-6.  ARE  ALL  TASK  ELEMENTS  INCLUDED?  Task  elements  are  component  behav- 
iors required  to  perform  the  task.  Elements  roughly  correspond  to  steps, 
behaviors,  or  actions  listed  In  the  task  analysis.  Ask  the  following 
questions  which  may  add  to  or  modify  the  elements,  depending  on  the  task. 

a.  What  Initiates  the  task?  If  the  task  Is  self-lnltlated,  consider 
Including  the  Initiating  cue  as  an  element. 

b.  What  completes  the  task?  If  the  end  of  the  task  Is  a condition 
or  product  requiring  Judgment  on  the  part  of  the  Incumbent,  consider 
Including  the  end  product  as  a stated  element. 

c.  Is  use  of  references  necessary?  If  a reference  (TH,  FM  or  other 
Job  performance  aid)  Is  used  normally  on  the  job,  that  fact  should  be 
Indicated  In  the  task  analysis.  References  necessary  for  task  perfor- 
mance on  the  Job  generally  will  have  to  be  available  for  test  purposes. 

This  availability  poses  no  problem  for  hands-on  testing,  but  may 
pose  a logistical  and  cost  problem  for  the  Written  Component  which  sust 
be  self-contained.  To  minimize  the  problem  ask,  "Is  the  ability  to  use 
the  reference,  or  the  ability  to  use  the  Information  In  the  reference, 
the  Important  part  of  the  task?" 
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If  ability  to  use  a manual  Is  Important,  this  should  be  treated  as  a 
separate  scorable  unit  within  the  hands-on  component*  If  ability  to  use 
Information  Is  the  Important  part,  "extract  and  Include  the  Information 
with  the  relevant  scorable  units  In  the  Written  Component*  Coordination 
with  ITED  Is  necessary  for  any  combination  of  extracts  exceeding  30  pages 
per  SQT* 

d*  Are  all  elements  stated?  Insure  that  a concise  statement  for 
each  step  Is  In  the  task  and  that  no  actions  have  been  left  out*  You 
may  not  use  all  the  steps  when  you  develop  performance  measures,  but  It 
Is  necessary  to  start  with  all  of  them* 


3-7*  ARE  NON-ELEMENTS  INCLUDED? 

a*  Once  you  are  sure  all  elements  have  been  Included  In  the  task 
analysis,  check  to  see  If  any  unnecessary  elements  are  listed*  Delete 
from  the  task  analysis  nonessentlal  elements*  For  example.  If  the  task 
analysis  data  on  target  engagement  Includes  knowledge  of  the  muzzle 
velocity  of  the  bullet,  that  knowledge  should  be  deleted  as  nonessentlal 
to  performance  of  that  task* 

b*  Finally,  mark  key  elements  on  the  task  analysis  data  form  by 
asterisks  or  other  means*  These  are  the  elements  on  which  to  focus 
attention  when  developing  scorable  units* 


3-8*  IS  THE  TASK  STANDARD  STATED? 

a*  The  task  analysis  should  include  a statement  as  to  standard  of 
performance*  Standards  state  how  well  a task  must  be  performed  and 
often  highlight  unacceptable  actions*  Standards  tell  how  to  Judge  the 
adequacy  of  performance  and  may  be  Included  as  an  element  In  the  analysis 
or  listed  separately*  Always  consider  possible  standards  of  sequence 
and  time* 


b*  Is  sequence  Important?  If  the  task  has  a required  sequence,  then 
the  sequence  must  be  stated*  Only  those  aspects  of  sequence  absolutely 
necessary  for  correct  performance  or  safety  to  soldier  or  equipment  must 
be  specified*  Be  alert  for  tasks  where  only  some  steps  must  be  done  In 
sequence*  Be  specific  when  describing  sequence*  For  example,  "All  steps 
must  be  performed  In  sequence,"  or  "Step  five  must  be  performed  last*" 

c*  Is  the  task  time  constrained?  If  the  task  Is  time-constrained 
as  a reflection  of  job  conditions,  the  time  standard  should  be  specified* 
Do  not  confuse  time-constrained  tasks  with  time  limits  routinely  stated 
In  the  SH,  or  that  may  be  adopted  later  for  test  administration* 

Examples  of  true  time  constraints  are  the  nine  seconds  to  put  on  the 
protective  mask,  or  two  minutes  to  remove  a misfire* 
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3-9.  ARE  KEY  ELEMENTS  IDENTIFIED? 


a.  Use  of  key  elements  is  unique  to  EQT  development.  So  it  is 
unlikely  that  key  elements  have  been  identified  during  task  analysis. 

b.  Key  elements  are  those  elements  which  have  been  identified  as: 

(1)  Being  most  common  sources  of  failure  in  performance 
of  the  task. 

(2)  Having  serious  consequences  of  failure. 

c.  The  first  aspect  of  key  elements  answers  the  question,  "Why  do 
most  soldiers  fall  to  perform  this  task  correctly?"  The  aspect  is  based 
on  frequency  or  the  "most  likely"  step  where  the  soldier  will  fall. 

The  aspect  may  be  based  on  data  gathered  during  evaluation  of  task  criti- 
cality (Chap.  2)  or  on  collection  of  opinions  from  subject  matter  experts. 

d.  The  second  aspect  of  key  elements  pertains  to  serious  conse- 
quences of  failure.  Many,  but  not  all,  elements  in  this  category  are 
related  to  personnel  or  equipment  safety.  The  elements  are  not  related 
to  frequency:  in  fact,  they  may  occur  very  Infrequently. 

But  consequences  of  their  occurring  are  so  serious  as  to  warrant 
testing  then. 

You  must  be  somewhat  selective  in  using  this  category  of  key  elements. 
Certainly  not  all  safety  warnings  qualify  for  Inclusion.  Information  on 
this  can  also  be  gathered  by  interviewing  subject  matter  experts. 

e.  Mot  every  task  trill  have  key  elements,  but  do  make  every  effort 
to  identify  them.  Developing  the  SQT  around  key  elements  will  help 
insure  that  testing  efforts  result  in  the  greatest  payoff. 

f.  At  the  end  of  this  chapter  is  a sample  questionnaire  that  may  be 
used  in  gathering  data  from  subject  matter  experts  about  task  elements 
and  key  elements  (Figure  3-2).  No  magic  formula  exists  for  analyzing 
the  data  on  key  elements  once  it  is  collected. 

Look  Instead  for  trends  or  consistencies  in  replies.  Most  experts 
will  identify  more  than  one  element  or  step  as  being  the  most  frequent 
reason  for  failure.  If  agreement  among  experts  is  good,  you  should  have 
no  problem  in  identifying  the  key  elements. 

On  the  other  hand,  if  agreement  is  poor,  you  may  have  to  follow-up 
by  discussing  sources  of  disagreement  %rlth  experts  in  attempting 
resolution.  Bear  in  mind,  however,  the  possibility  that  no  one  or  two 
elements  can  be  pinpointed  as  common  sources  of  failure. 
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g.  Significance  of  key  elements  will  be  obvious  when  you  develop 
scorable  units,  particularly  written  scorable  units.  Vftien  tasks  have 
many  elements,  concentrate  on  preparing  items  that  focus  on  key 
elements. 

Additionally,  several  written  items  likely  will  be  developed  around 
key  elements  to  insure  at  least  one  Item  per  key  element  survives 
validation. 


3-10.  DO  STATEMENTS  PERMIT  IDENTIFICATION  OF  INCORRECT  PERFORMANCE? 

a.  After  key  elements  have  been  Identified,  examine  each  element 
statement  to  determine  whether  an  adequate  basis  exists  fot  identifying 
specific  errors  common  in  Incorrect  performance.  Often  the  statements 
are  too  general  to  be  of  much  use. 

Ask  yourself,  "What  does  this  statement  mean?"  and,  "Can  I tell  from 
this  statement  if  a soldier  is  correctly  or  incorrectly  performing  this 
step?"  Add  to  the  statement,  if  necessary,  a standard  or  way  to  measure 
the  performance. 

Compare,  for  example,  "adjusts  volume"  with  "turns  volume  knob  to 
LOUD  position."  Compare  "cleans  metal  surfaces"  with  "removes  all  traces 
of  dirt  and  rust,  using  solvent  and  #2  emory  paper.  Wipes  dry  with  clean 
soft  cloth.  Applies  light  coat  of  PLL  special  and  again  wipes  surface 
dry." 

b.  Statements  that  do  not  allow  identification  of  incorrect  perfor- 
mance must  be  revised.  While  TM,  FM,  and  other  publications  will  provide 
you  much  needed  information,  it  may  be  necessary  to  seek  subject  matter 
expertise.  Insure  statement  accuracy  by  checking  with  several  Job  and 
subject  matter  experts. 
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Sample  Questionnaire 

IDENTIFYING  KEY  ELEMENTS  IN 
TASK  PERFORMANCE:  QUESTIONS  FOR  A JOB  EXPERT 

1.  la  the  task  analyala  a coeplete  Hating  of  all  eleaenta  (declalona  or 
actlona)  neceaaary  for  aucceaaful  taak  perforaance? 


If  not,  what  eleaenta  should  be 


Added?  Please  List: 


b<  Deleted?  Please  list: 


c.  Revised?  Please  Hat: 


2«  You  have  seen  this  task  perfoiraed  asny  tines  In  the  field.  Of  the 

tines  you  have  seen  soldiers  fall  to  perfora  It  successfully,  why  have 
they  failed?  Where  do  they  usually  go  wrong?  Please  consider  the 
original  list  of  task  elencnts,  plus  any  revisions  you  nay  have  aade, 
and  Identify  the  eleaent  that  la  the  aost  frequent  cause  of  failure. 

Most  frequent  cause  

Are  there  other  task  eleaents  that  stand  out  In  your  aenory  as  fre- 
quent causes  of  failure?  If  so,  please  list  then. 


Soaetlaes  perforaance  of  a task  Involves  an  eleaent  that  is  v»ry  In- 
portant  In  the  sense  that  failure  to  perfora  that  eleacnl  can  lead  to 
serious  consequences,  serious  injury  to  the  soldier,  or  serious  dan- 
age  to  equlpnent.  Is  auch  an  eleaent  part  of  the  task  being  reviewed? 


If  there  Is,  what  Is  it? 


Figure  3-2.  Sarnplt  Questionnaire 
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CHAPTER  4 

ALLOCATE  TASKS  TO  SQT  COMPONENTS 


Section  I.  TASK  ALLOCATION 

4-1.  INTRODUCTION. 

a.  After  tasks  are  selected  for  testing  and  review  of  tasks  analysis 
Inforaatlon  Is  completed*  the  next  step  Is  to  allocate  each  task  (or  In 
some  cases,  part  of  a task)  to  one  of  three  components  (HOC*  WC*  and  PCC) 
of  the  SQT.  Basic  Information  Inputs  to  use  In  the  allocation  process 
are: 

(1)  A list  of  tasks  roughly  ordered  as  to  Importance 
for  testing. 

(2)  A list  of  elements  for  each  task  with  key  elements 
Identified. 

b.  Since  tasks  will  be  grouped  with  respect  to  Importance  or  criti- 
cality* the  general  procedure  is  to  start  at  the  top  of  the  list  and  go 
from  "most"  Important  to  "least"  important  until  the  allocation  process 
Is  complete.  When  there  are  a large  number  of  tasks*  this  process  will 
Insure  that  the  Important  tasks  are  Included  In  the  SQT. 

c.  Two  basic  concepts  govern  the  process  for  allocating  tasks  to 

SQT  components.  Concepts  are  task  content  and  testing  feasibility.  Both 
concepts  assume  that  you  know  how  the  task  being  considered  Is  performed. 

If  you  have  gone  through  requirements  In  Chapter  3*  Reviewing  Task 
Analysis*  you  will  know  how  the  task  Is  performed*  even  though  you  may 
never  have  seen  the  task  performed.  If  unsure  about  part  of  task  perform- 
ance* check  with  a subject  matter  expert.  Keep  in  mind  that  concepts 
may  be  applied  to  different  parts  of  the  task. 


(1)  Task  Content. 

(a)  This  concept  Involves  deciding  on  the  type  of  task  and  the 
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(b)  Think  of  physical  tasks  as  those  that  require  use  of  hands, 
arms,  legs,  or  body  In  performance.  Think  of  mental  tasks  as  those 
that  require  primarily  use  of  the  reasoning  or  thinking  process  In 
performance.  Be  aware  that  considerable  overlap  exists. 

A.11  physical  tasks  require  some  degree  (often  very  high)  of 
mental  activity.  All  mental  tasks  require  some  physical  activity.  If 
only  pushing  a button  or  writing  something.  Be  aware  of  the  purpose 
of  the  classification  of  tasks  by  type  - to  decide  what  Is  needed  for 
measurement . 

Ask  yourself,  "If  I watched  a soldier  perform  this  task,  what 
would  I observe?"  If  there  Is  a lot  of  physical  activity,  the  task  is 
probably  physical.  If  most  of  the  essential  activity  cannot  be  observed 
(because  It  goes  on  In  the  soldier's  head),  the  task  Is  probably  mental. 

(c)  Likewise,  do  not  confuse  skill  with  ability  or  level  of  the 
soldier.  Skill  (high  or  low)  has  to  do  with  how  much  practice  Is  needed 
to  become  proficient  In  the  task — mental  or  physical. 

If  a soldier  can  read,  or  be  told,  about  a task  and  then  perform 
It  with  no  more  than  one  practice  trial,  the  task  Is  probably  low  skill. 
The  more  complex  a task  becomes,  the  higher  the  skill  generally  required. 

(d)  Certainly  a degree  of  Judgment  Is  required  In  the  above 
decisions,  so  do  not  be  concerned  with  precise  labeling  of  all  tasks. 
Also,  you  may  change  your  mind  about  tasks  or  subdivide  parts  of  tasks 
Into  different  classifications. 

Be  cautious  about  subdividing.  Don't  try  to  break  a task  into  too 
many  categories  - you  will  end  up  with  something  probably  untestable. 

Keep  In  mind  the  reason  for  the  concept  of  task  content.  All  types  and 
levels  of  skill  can  be  tested  validly  In  a hands-on  mode,  assuming 
unlimited  resources. 


Some  physical  tasks  may  be  tested  in  the  written  component,  but 
highly  skilled  physical  tasks  must  be  tested  In  the  hands-on  component 
or  performance  certification  component.  Some  mental  tasks  may  need  to 
be  tested  In  the  hands-on  component,  but  mental  tasks  are  usually  more 
amenable  to  variations  that  permit  written  testing. 


(2)  Testing  Feasibility. 


(a)  Testing  requires  some  resources.  In  the  case  of  SQT,  the 
written  component  Is  the  least  demanding  since,  other  than  occasional 
use  of  Job  aids  and  AV  media.  It  requires  only  paper  and  pencil.  More- 
over, many  soldiers  can  be  tested  on  many  tasks  In  a relatively  short 
time  with  one  test  administrator. 
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The  hands-on  component,  however,  requires  equipment,  terrain,  and 
several  qualified  scorers.  In  addition,  relatively  few  tasks  can  be 
tested  In  four  hours. 

The  performance  certification  component  was  set  up  to  accomodate 
tasks  that  are  extremely  time  consuming  and  that  require  special  terrain, 
facilities,  or  expensive  equipment. 

(b)  With  an  understanding  of  the  above  concepts,  take  each  task 
through  the  flow  chart  at  the  beginning  of  this  section. 


4-2.  IS  ANY  PART  OF  THE  TASK  HIGHLY  SKILLED? 

a.  "Any  part"  refers  to  blocks  of  elements  - not  Just  single  ele- 
ments unless  It  applies  to  single  elements  Identified  as  key  elements 
(see  Chapter  3 for  Identification  of  key  elements). 

b.  "Highly  skilled"  should  be  thought  of  In  terms  of  how  much 
hands-on  practice  Is  requited  to  become  proficient  at  the  task.  Or, 
from  the  opposite  point  of  view,  can  the  task  be  performed  by  soldiers 
who  know  the  procedure  even  though  they  actually  may  have  done  it  only 
once  or  twice?  Some  useful  clues  to  Identifying  high-sklll  physical 
tasks  are: 

•Task  Involves  finely  tuned  acts  of  physical  coordination, 
e.g.,  tracking  a target,  engaging  targets  with  an  M16. 

•Task  requires  quick  reaction;  must  be  performed  with  tight 
time  limit,  e.g.,  clearing  a jammed  weapon,  typing  a 
message,  donning  a gas  mask. 

•Task  Involves  time-sharing  - doing  two  things  at  once, 
e.g.,  tactical  driving. 

c.  If  no  parts  of  the  task  Involve  highly  skilled  actions,  the  task 
should  be  considered  for  the  WC. 


4-3.  IS  THE  TASK  ESSENTIALLY  MENTAL?  If  a part  of  the  task  Is  highly 
skilled,  determined  whether  the  skill  Is  primarily  mental  or  physical. 

Is  the  skilled  part  of  the  task  a mental  operation  or  a series  of  mental 
operations  for  which  the  physical  process  or  outcome  Is  secondary? 

For  example,  performing  a numerical  calculation,  understanding  and 
selectively  applying  a rule  or  principle,  and  Identifying  objects  by 
name.  If  the  skill  Is  essentially  mental,  the  task  should  be  considered 
for  the  written  component. 
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4-4.  CAN  ENTIRE  TASK  BE  TESTED  FEASIBLY  IN  HOC?  Doe*  tlae,  equlpnent, 
terrain,  facilities,  or  availability  of  qualified  scorer*  dictate  the 
Inposslblllty  of  testing  the  task  In  the  hands-on  component?  If  feasible 
to  test  the  task  In  the  hands-on  component.  It  should  be  considered  for 
this  component. 


4-5.  CAN  THE  SKILLED  PART  BE  TESTED  FEASIBLY  IN  THE  HOC?  If  the  entire 
task  cannot  be  tested  feasibly  in  the  hands-on  component,  perhaps  you  can 
evaluate  one  or  more  of  the  key  skilled  elements  by  changing  the  task 
limits. 

For  example,  you  cannot  have  soldiers  fire  the  LAW  In  a hands-on  compo- 
nent, but  you  can  have  them  place  the  LAW  In  operation.  All  highly 
skilled  physical  elements  that  can  be  tested  feasibly  In  the  hands-on 
component  should  be  considered  for  allocation  to  this  component. 

Other  parts  or  elements  of  the  task  may  be  primarily  mental  or  repre- 
sent an  unskilled  requirement  In  the  performance.  These  elements  should 
be  tested  also  and  be  considered  for  allocation  to  the  written  component. 

If  not  feasible  to  test  the  skilled  part  of  the  task  In  the  hands-on 
component,  or  to  change  task  limits  to  allow  meaningful  testing,  place 
the  task  In  the  PCC. 


4-6.  COMPONENT  LIMITS. 

a.  As  tasks  are  allocated  to  components,  some  components  may  start  to 
fill  up.  Remember  the  limits  on  the  size  of  the  components: 

Hands-On:  0-16  SU  for  SQT  2 or  3;  0-10  SU 
for  SQT  4 and  5. 

Written:  30-50  SU. 

Performance  Certification:  0-10  SU. 

TOTAL  SQT:  30-76  SU 

b.  The  purpose  of  the  grouping  (Chapter  2)  was  to  Insure  that  "anst 
Important"  tasks  were  considered  for  testing  first.  When  a component 
fills  up  during  the  allocation  process,  check  each  additional  task 
against  a task  (usually  the  last  one)  already  In  the  component  and  decide 
which  Is  more  Important  for  testing  In  that  component. 
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Section  II.  PREPARE  SQT  PLAN 

4-7.  After  tasks  have  been  allocated  to  components,  the  SQT  Plan  must 
be  submitted  to  ITED.  An  outline  for  the  content  of  the  SQT  Plan  Is 
attached  (Figure  4-2). 


I 4-8.  If  the  Soldier's  Manual  for  the  MOS  has  not  been  printed,  send  a 

f developmental  draft  of  the  SM  with  the  SQT  Plan. 

I 
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level  being  tested. 
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CHAPTER  5 

CONSTRUCT  HANDS-ON  COMPONENT  SCORABLE  UNIT 
OVERVIEW 

The  purpose  of  this  chapter  Is  to  provide  a procedure  for  constructing 
a complete  hands-on  SU.  The  end  result  Is  a standardised  method  of 
measuring  a critical  part  of  what  the  soldier  does  on  the  Job* 

1.  Basically*  five  areas  have  to  be  developed  In  writing  a hands-on 
SU.  The  five  areas  will  ansrnr  five  essential  questions. 

a.  What  la  going  to  be  scored? 

This  la  answered  by  deciding  %diether  the  test  will  be  process  or 
product . 

b.  How  will  the  soldier's  performance  be  scored? 

This  Is  answered  by  developlna  process  or  product  performance 
measures. 


c*  How  will  the  test  be  set  up? 

This  Is  anstrered  by  developing  test  conditions  for  the  scorable 
unit . 

d.  How  will  the  soldier  know  what  to  do  on  the  test? 


This  Is  answered  by  preparlna  Instructions  to  the  examinee, 
e.  How  will  the  scorer  know  what  to  do  on  the  test? 


This  Is  answered  by  preparing  Instructions  to  the  scorers. 

2.  As  with  any  area  of  test  development,  the  starting  point  Is  the 
task  analysis.  This  will  tell  you: 

a.  What  you  are  going  to  test  (the  task). 

b.  What  the  soldier  must  do  to  perform  the  task  (elements). 
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3.  Refine  the  information  from  task  analysis  into  a hands-on  test* 
During  this  procedure,  you  may  find  faults  with  the  task  analysis  or 
Information  from  the  task  analysis  that  will  not  fit  your  needs  for 
hands-on  development*  These  must  be  resolved  as  they  arise* 

4*  Concentrate  on  the  production  of  good  reliable  performance 
measures  (product  or  process)*  During  development  and  refinement  of 
these  performance  measures,  keep  in  mind  what  condition  the  equipment 
must  be  In,  what  guidelines  the  scorer  will  need,  and  what  Instructions 
the  examinee  will  require* 

If  you  do  this  and  maintain  an  Informal  note  system  as  you  refine  the 
performance  measures,  the  test  conditions,  examinee  Instructions  and 
scorer  Instructions  will  essentially  be  written  by  the  time  performance 
measures  are  completed* 

5*  As  part  of  the  hands-on  package,  each  TDA  will  prepare  scorer 
training  tapes,  a scorer  training  package  for  each  test  station  and 
audio-visual  tapes  for  examinee  Instructions* 

6*  As  you  go  through  this  process,  remember  what  you  are  trying  to 
develop*  That  Is: 

a*  A test  that  measures  what  the  soldier  has  to  do  on  the  Job* 

b*  A test  that  can  be  administered  to  a soldier  In  many  different 
locales  under  standardized  conditions* 

c*  A test  that  can  be  scored  fairly  and  accurately  wherever  and 
by  whomever  It  Is  administered* 


Section  I*  DETERMINE  PRODUCT  OR  PROCESS 


5-1*  INTRODUCTION* 


a*  The  first  step  In  developing  a hands-on  test  Is  to  determine 
w:hether  It  Is  a process-  or  a product-scored  task*  To  determine  this, 
decide  what  the  scorer  must  observe  to  evaluate  the  soldier's  perform- 
ance* Must  the  scorer  observe  every  element  in  the  task  performance 
or  can  the  scorer  focus  on  some  product  produced  during  the  performance 
of  the  task?  A simple  example  should  serve  to  highlight  the  difference* 


Combine  product 
■nd  proccw 
(per.  5-6) 


product 
(pw.  64) 


•t 
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Perfora  an  about-face  Is  a process  task.  (The  scorer 
aust  observe  the  entire  aovement  while  It  Is  being 
perforaed  to  deteralne  adequacy  of  perforaance.) 

Tie  a square-knot  Is  a product  task.  (The  scorer  would 
not  even  have  to  be  present  while  the  task  Is 
perforaed.  Assualng  adequate  controls,  the  scorer 
can  evaluate  the  perforaance  anytlae  It  Is  coapleted.) 

b.  The  goal  Is  to  achieve  product-type  tests  or  aeasureaents  If 
possible,  since  they  simplify  test  adalnlstratlon.  Product  scoring 

Is  generally  aore  standardized,  does  not  require  detailed  concentration 
on  the  part  of  the  scorer  and,  for  some  tasks,  can  allow  more  than  one 
Individual  to  be  tested  at  the  same  time. 

c.  Differences  between  product  and  process  become  more  subtle  as 
Individual  tasks  are  considered  for  testing. 

Indeed  you  aust  understand  the  process  of  any  task  before  a process/ 
product  decision  can  be  ahde.  Most  tasks  will  be  thought  of  Initially 
as  process  tasks  because  they  will  appear  that  way  In  the  SM. 

A task  Is  usually  taught  as  a series  of  steps  (elements);  therefore 
developers  may  tend  to  think  of  most  tasks  as  process.  Now  the  emphasis 
Is  on  testing,  and  many  tasks  that  are  outwardly  process.  In  fact,  can  be 
product-scored  during  testing. 

d.  There  Is  usually  no  problem  In  Identifying  products  that  are  fabri- 
cated; that  Is,  something  produced  something  that  we  can  hold  In  our 
hands.  Most  "products"  of  tasks,  however,  are  not  that  distinctive  or 
tangible.  Many  products  will  be  actions  or  results.  Some  examples  are: 

A radio  check  (audio)  for  placing  a radio 
set  Into  operation. 

A weapon  function  check  for  assembling  a 
pistol. 


5-2.  DOES  THE  TASK  PRODUCE  AN  OBSERVABLE  PRODUCT?  You  aust  decide  If 
the  end  result  of  a task  meets  the  definition  of  a product.  You  should 
be  able  to  state  In  a few  words  what  the  end  result  Is  and  then  decide  If 
this  Is  aeasurable  as  a product. 

Products  allow  us  to  aake  certain  assuaptlons  about  the  process  of 
their  production  even  though  this  was  not  observed.  If  the  task  has  no 
product.  It  aust  be  scored  as  a pure  process  test. 
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5-3.  ARE  THERE  ELEMENTS  NOT  EVIDENT  IN  THE  PRODUCT  THAT  MUST  BE  SCORED? 
Even  If  you  decide  that  the  task  has  an  observable  product*  exaalne  the 
task  analysis  to  Identify  %dilch  elements  of  the  task  are  measurable  In 
the  product,  and  vhlch  are  not.  The  following  factors  should  be  consid- 
ered In  Identifying  elements  that  cannot  be  scored  from  the  product: 

a.  Safety  factors.  Safety  will  be  as  Important  a concern  In  testing 
situations  as  It  la  on  the  Job.  Safety  factors  Include  actions  that 
could  lead  to  personal  Injury  or  damaged  equipment,  and  are  Identified 
as  key  elements  In  the  task  analysis. 

Safety  factors  must  be  specifically  stated.  They  require  vigilance 
on  the  part  of  the  scorer  to  prevent  violations.  Not  all  safety  consid- 
erations will  be  Included;  only  those  Identified  as  key  elements. 

Safety  considerations  become  process-tested  measures  since  they  usu- 
ally require  the  scorer  to  Intervene  In  the  test  to  prevent.  If  possible, 
violations  from  occurring. 

b.  Elements  hidden  by  the  product  Itself.  Some  process  elements 
become  physically  obscured  by  the  product  or  end  result  to  the  extent 
that  they  cannot  be  determined  readily  by  outward  examination  of  the 
product.  Some  product  assembly  or  fabrication  tasks  are  of  this  type. 

More  common  are  tasks  that  Involve  a product  but  which  contain  elements 
that  cannot  be  evaluated  accurately  from  the  product  because  they  are 
not  lasting.  For  example,  clearing  the  protective  mask  or  turning  off 
radios  before  starting  a vehicle. 

c.  Diagnostic  evaluation.  You  must  anticipate  the  requirement  for 
feedback  of  test  results.  "Pure"  product  tests  will  Identify  soldiers 
who  cannot  perform  the  task,  but  may  not  reveal  which  elements  they 
fall  most  frequently. 

Therefore,  when  considering  a product  test,  you  must  decide  If  a 
a requirement  exists  for  diagnostic  feedback.  If  there  Is,  you  must 
provide  for  scoring  of  elements  In  order  to  provide  that  feedback. 

Often  this  will  be  a matter  of  policy  and  should  be  coordinated  with 
Directorate  of  Evaluation  or  other  group  responsible  for  evaluating 
and  reporting  post-test  Information  In  the  TDA. 


i 
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5-4.  MEASURE  PRODUCT.  If  all  elesMnts  can  be  measured  accurately  from 
the  product  or~end  result,  the  test  will  become  a "pure"  product  test, 
and  you  need  only  describe  the  product.  How  to  do  this  will  be  detailed 
later. 
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5-5.  COMBINE  PRODUCT  AND  PROCESS.  Many  tasks  will  not  fall  noatly 
Into  the  product  or  the  process  classification.  Therefore  the  term, 
"Combination  Process/Product"  will  apply  to  many  tasks. 

In  the  combination  of  the  two  types,  you  first  Identify  those  elements 
that  must  be  evaluated  during  the  process  of  performance,  then  use 
product  measures  for  the  remainder  of  the  task.  Even  In  process  tests, 
the  scorer  should  be  directed  to  elements  where  measurement  is  best 
^ made . 

j Section  II.  SPECIFY  PERFORMANCE  MEASURES:  PROCESS 


5-6.  INTRODUCTION.  In  this  part,  you  must  tell  the  scorer  what  you  want  ! 

him  to  look  at  and  how  you  want  him  to  Judge  what  he  sees.  The  starting  ) 

point  for  process  measures  are  elements  as  listed  In  the  task  analysis. 

Now,  refine  those  statements  so  they  are  presented  In  a way  that 
can  be  scored  accurately  and  consistently  by  many  different  scorers. 

Remember,  In  process-scoring  we  are  looking  at  what  the  soldier  Is  doing  t 

as  he  Is  doing  It.  j 


5-7.  ARE  PERFORMANCE  STANDARDS  STATED?  For  many  elements  (performance 
measures).  It  Is  Insufficient  to  simply  state  the  action  because  you 
believe  the  standard  is  implied  In  the  element.  During  review  of  task 
analysis,  you  attempted  to  determine  If  the  element  was  described  In 
sufficient  detail  so  that  Incorrect  performance  could  be  determined. 

At  this  point,  you  are  double-checking  to  see  that  the  standard  Is 
specified  for  each  performance  measure.  Standards  answer  questions 
relating  to  "how  much"  or  "how  well."  You  should  look  carefully  and  ask 
questions  about  terms  like  "turn,"  "adjust,"  "tighten,"  "loosen,"  and 
other  words  that  Imply  Judgment  about  whether  the  step  is  performed 
correctly. 

Likewise,  be  wary  of  the  "ly"  adverbs  In  describing  standards.  Stan- 
dards using  terms  such  qs  "properly,"  "rapidly,"  "firmly,"  "cccurately," 
are  often  unmeasurable  and  may  be  scored  differently  by  different 
scorers. 


5-8.  IS  MORE  THAN  ONE  PASS/FAIL  PERFORMANCE  MEASURE  STATED? 

a.  Each  performance  measure  should  spell  out  a distinct  action. 

Avoid  confusing  the  scorer.  For  example:  "Removes  clip  snd  pulls  slide 
to  the  rear,"  as  a performance  measure,  may  be  confusing  to  a scorer. 
What  about  the  soldier  who  removes  the  clip  but  does  not  pull  the  slide 
to  the  rear? 
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You  are  aaklng  the  scorer  to  Bake  a Judgment  on  which  was  more  impor- 
tant* Will  he  score  It  the  way  you  want  hla  to?  To  Insure  standardized 
scoring,  separate  any  performance  measures  that  are  lumped  together. 

b*  Simultaneous  actions  are  different.  “Depresses  accelerator  while 
engaging  starter,"  la  an  acceptable  statement.  So  are  statements  of  how 
a step  Is  performed,  such  as  "alines  sight  by  traversing,  elevating  or 
depressing  gun  tube."  Oftep  such  statements  will  be  needed  to  cue  the 
scorer  to  the  actions  performed. 


5-  9.  ARE  ONLY  NECESSARY  ACTIONS  LISTED? 

a.  After  breaking  out  statements  Into  separate  elements,  you  may  have 
too  suiny  actions  listed.  Some  actions  need  not  be  stated  If  they  are 
dependent  on  other  actions. 

For  example:  "releases  safety  cover;  activates  firing  switch."  Assum- 
ing the  firing  switch  cannot  be  activated  without  releasing  the  cover, 
eliminate  the  first  element  from  scoring. 

b*  It  may  not  be  always  necessary  to  list  every  element  from  the  task 
analysis  as  a performance  measure  on  the  test.  Remember  the  machine 
scoreaheet  Is  limited  to  pass/fall  for  20  Items*  No  exact  rules  can  be 
stated  for  elimination  of  elements  If  over  20  are  listed. 

Developers  must  work  with  subject  matter  experts  to  direct  the  scorer 
to  that  point  where  measurement  is  best  made  but  where  no  questloa 
remains  as  to  whether  a step  was  performed  correctly  or  Incorrectly.  You 
must  be  careful  not  to  overwhelm  the  scorer  with  performance  measures; 
the  scorer  cannot  observe  the  soldier  while  he  reads  and  marks  the  score- 
sheet. 

Some  elements  listed  may  be  performed  In  the  task  but  are  not  essen- 
tial to  measure.  Try  to  direct  the  scorer  to  that  point  where  measure- 
ment Is  best  made.  The  goal  Is  to  make  the  test  easy  to  score. 


5-10.  IS  ACTION  OBSERVABLE? 

a.  Is  the  behavior  observable  as  a physical  act?  An  eye  or  head  move- 
ment may  not  be  dlscernable  as  a physical  act.  For  example:  "Throttle 
Is  In  Idle  position,"  can  be  evaluated  only  If  the  throttle  Is  placed  s.ii 
another  (Incorrect)  position  and  the  soldier  has  to  move  It  to  the  Idle 
position. 

Likewise,  "look  to  see  If  backblast  area  la  clear"  could  be  evaluated 
by  placing  something  In  the  backblast  area  (assuming  this  can  be  done 
realistically  and  safely  In  a test  condition). 
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Soae  caution  la  necessary  when  preparing  the  scoresheet.  Soaetlaes 
you  must  use  parenthetical  notes  to  the  scorer  as  to  how  he  evaluates 
the  aeasure  of  "checking*" 

b*  Some  anblguous  actions  aay  be  the  result  of  poor  wording  of  a 
performance  aeasure*  For  example:  "Checks  radar  scope"  may  actually 
mean  the  soldier  has  to  turn  and  look  at  the  scope*  Cue  the  scorer 
to  the  performance  by  stating,  "turns  and  looks  at  scope  #1  before  acti- 
vating scanner*" 


5-11*  ARE  ERROR  TOLERANCE  LIMITS  ALLOWED?  Error  tolerance  limits  allow 
a range  of  accuracy  on  some  standards*  Such  limits  are  often  needed  In 
dealing  with  standards  Involving  measurements*  Avoid  a specific  number 
unless  that  number  Is  the  actual  standard* 

Most  actions  tflll  allow  a range;  for  example,  "700-800  rpm,"  "within 
50  meters,"  "+  5°."  Specify  the  range  allowed*  If  appropriate,  specify 
how  the  measurement  Is  made  or  tfhat  the  mean  reference  point  Is* 


5-12.  ARE  SPECIFIC  SAFETY  MEASURES  STATED? 

a*  If  safety  la  a factor,  write  a safety  performance  measure*  This 
performance  measure  cites  a specific  action  must  or  must  not  be  done* 

The  measure  should  cause  the  scorer  to  Intervene  In  the  testing  to 
prevent  the  violation  If  possible  (whether  It  should  cause  a NO-GO 
score  Is  up  to  the  TDA). 

b*  Statements  such  as  "commits  no  safety  violations,"  are  unspecific, 
unstandardised  and  unacceptable* 


5-13*  IS  TIME  LIMIT  STATED? 

a*  If  the  task  Is  tlme-coistralned,  the  time  limit  Is  a performance 
measure*  For  example,  "Mask  in  9 seconds*"  Insure  the  start  and  stop 
points  are  clearly  Identified,  both  to  the  scorer  and  the  examinee* 

Time-constrained  tasks  are  identified  usually  as  such  In  the  task 
analysis*  If  you  have  altered  the  test  limits.  Insure  the  time  Is 
adjusted  accordingly* 

b*  If  the  task  Is  not  time-constrained,  a time  limit  must  still  be 
assigned  to  enable  efficient  test  administration*  Utilize  Job  experts 
and  field  tryouts  to  arrive  at  reasonable  time  limits* 
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c.  As  a final  consideration,  make  an  estimate  of  the  time  the 
scorable  unit  will  take*  Look  at  performance  tlsw,  plus  setup  time, 
scoring  time,  and  examinee  briefing  time.  This  la  the  total  adminis- 
tration time* 

You  may  have  guidance  on  maximum  allowable  time  for  an  SU*  If  not, 
look  closely  at  any  Item  exceeding  30  minutes*  The  entire  component 
must  be  administered  in  four  hours*  If  tlaw  estimates  are  accurate  and 
the  SU  Is  too  long,  you  may  need  to  tighten  the  test  limits  some* 


5-14*  IS  SEQUENCE  STATED? 

a*  Examine  the  task  elements  carefully  for  sequence  requirements* 
Some  tasks  require  total  sequence,  others  only  partial  sequence*  If 
sequence  Is  required,  state  It  as  a performance  measure* 

b*  If  no  sequence  Is  required,  state  this  for  the  scorer  on  the 
scoresheet* 


Section  III*  SPECIFY  PERFORMANCE  MEASURES:  PRODUCT 


5-15*  INTRODUCTION*  In  this  block,  you  must  decide  what  the  scorer 
needs  to  look  at  when  he  has  a product  teat  (either  as  pure  product 
test  or  Included  In  a combination  process/product  test)  In  order  to 
evaluate  a soldier's  performance* 

Much  of  what  was  already  discussed  In  process  testing  also  applies 
here*  The  scorer  la  concerned  here  In  looking  at  something  a soldier 
has  done,  not  In  watching  him  do  It*  This  should  both  ease  the  Job  of 
the  scorer  and  make  his  evaluation  more  accurate* 

But  It  places  a greater  burden  to  Insure  that  you  precisely  define 
the  correct  outcome*  Though  the  starting  point  Is  again  task  analysis,  j 

products  often  are  not  clearly  evident  or  defined  in  the  task  analysis* 

Look  for  end  results  - not  how  they  were  achieved* 


3-16.  DEFINE  ACCEPTABLE  PRODUCT* 

a*  This  equates  with  the  standard  of  performance*  It  may  have  Identi- 
fiable characteristics  or  It  may  be  descrlbable  only  by  the  absence  of 
unacceptable  characteristics* 

For  example,  the  standard  (or  acceptable  product)  for  typing  correspon- 
dence may  be  that  It  follows  a specified  format  and  contains  no  strike- 
overs  or  misspelled  words* 
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Figure  54.  Specify  Performance  Meawree:  Product 
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b.  Check  for  the  standard  in  the  taak  analysis,  but  remember  it  may 
not  be  readily  apparent*  Most  task  analyses  are  prepared  in  a procedural 
format.  If  you  cannot  deduce  a standard  for  the  product  from  the  taak 
analysis,  consult  with  subject  matter  experts. 

Remember  an  "acceptable  product"  nay  be  a condition  such  as  a running 
engine  or  ringing  telephone;  it  nay  be  an  action  such  as  a weapon  func- 
tion check;  or  bonethlng  produced  or  constructed  such  as  a completed 
form  or  a computer  program  measurable  by  description  or  by  comparison 
with  a model. 


5-17.  DEFINE  OBSERVABLE  STANDARD  FOR  EACH  DIMENSION  OF  THE  PRODUCT. 

a.  Carefully  examine  products  for  all  dimensions  and  define  all  the 
standards  necessary  to  assist  in  scoring.  Examples  of  various  dimensions 
are: 


-height 

-width 

-depth 

-length 

-format 

-texture 


-tightness 

-strength 

-elasticity 

-precision 

-tolerance 


-shape 

-color 

-location 

-position 

-sound 

-rhythm 


-accuracy 

-time 

-speed 

-size 

-adjustment 


b.  Insure  that  each  dimension  is  specified  in  a manner  that  is  observ- 
able or  measurable*  Remember,  most  measures  or  standards  must  be  suit- 
able for  rating  by  unsophisticated  methods. 


5-18.  IS  PERFORMANCE  TIMED? 

a.  Some  product  tests  are  time-constrained.  Time  is  usually  speci- 
fied as  a standard  in  the  task  analysis.  Time-constrained  tasks  usually 
will  Include  the  time  as  part  of  the  product  standard.  Start-and-stop 
points  must  be  clearly  defined.  The  stop  point  (or  limit)  also  must 

be  clearly  defined  to  the  soldier. 

b.  If  performance  is  not  time-constrained,  a time  limit  must  be 
established  to  Insure  efficient  test  administration.  This  time  limit 

is  not  meant  to  be  discriminatory,  l.e.,  the  soldier  who  can  perform  the 
task  does  so  easily  within  the  time  limit,  the  soldier  who  cannot  likely 
falls  for  reasons  other  than  time.  Determine  performance  times  with  sub- 
ject matter  expert  and  field  tryouts. 


j 


I 
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c.  Finally,  consider  the  overall  time  required  for  administering  the 
scorable  unit.  Add  performance  time,  briefing  time,  setup  time,  and 
scoring  time.  Remeaiber,  the  component  must  be  administered  In  four 
hours . 


You  may  have  received  guidance  on  how  long  an  SU  can  be.  If  not,  look 
carefully  at  any  SU  where  the  total  time  exceeds  30  adnutes.  You  nay 
want  to  redefine  the  test  limits  to  shorten  the  required  time. 
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5-19.  SPECIFY  TOLERANCE  FOR  EACH  STANDARD.  If  a range  of  measurement  la 
acceptable  In  the  standard,  insure  that  the  range  Is  specified.  Few 
products  or  end  results  that  involve  a numerical  measurement  Involve  a 
specified  number.  If  a range  is  allowed,  insure  that  range  is  specified; 
for  example,  "50-70  FPs  torque,"  "within  50  meters." 


5-20.  CAN  SCORING  AIDS  BE  USED? 

a.  Scoring  aids  are  used  to  help  standardize  what  is  being  scored. 
Consider  the  use  of  the  following  kinds  of  aids: 

-overlays  -diagrams 

-templates  -photographs  of  correct  product 

-blueprints  -photographs  showing  unacceptable 

-models  characteristics 

b.  Scoring  aids  are  not  easy  to  prepare  but  should  be  considered 
where  mere  words  do  not  convey  an  accurate  understanding  of  the  standard 
or  where  use  of  aids  will  speed  up  the  scoring  process.  For  example: 
the  tasks  "weld  a bead,"  "prepare  route  overlay,"  "wire  a circuit,"  are 
products  difficult  to  describe  in  words  but  clearly  shown  by  example. 

c.  If  scoring  aids  are  used,  they  must  be  produced  in  a quantity 
sufficient  to  test  the  MOS  worldwide.  Therefore,  you  must  be  sure  that 
the  aids  are  reproducible  and  can  be  readily  distributed.  Coordinate 
with  ITED  when  scoring  aids  are  planned. 


5-21.  INSURE  PRODUCT  IS  PRESERVED. 

a.  The  product  need  be  preserved  only  until  evaluated  and  a record 
(score)  made.  In  most  cases,  this  action  will  be  immediately  after 
production.  However,  some  products  do  not  last  long,  e.g.,  the  ringing 
of  a telephone  or  a communications  check. 

Others  are  easily  destroyed,  e.g.,  a sight  allnement,  a compression 
reading  or  a compass  reading.  Insure  that  instructions  to  the  scorer 
Include  procedures  in  how  and  when  to  record  scores  for  products 
of  short  duration  or  that  are  easily  destroyed. 

b.  Some  products  may  be  preserved  for  later  evaluation  by  tagging, 
recording,  or  even  videotaping.  Be  sure  a positive  method  for  identi- 
fying each  product  is  provided. 


I 
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Section  IV.  DETERMINE  TEST  CONDITIONS 


5-22.  INTRODUCTION. 

a.  In  this  block,  you  oust  determine  what  the  soldier  Is  going  to 
experience  (see,  hear,  feel)  when  he  takes  the  test.  We  are  concerned 
primarily  with  the  effect  these  factors  will  have  on  his  performance  on 
the  tes*" 

In  effect,  you  are  telling  the  scorer  how  to  set  up  the  test  so  the 
soldier  will  take  the  test  the  way  you  visualized  It.  Start  with  the 
statement  of  conditions  In  the  task  analysis.  However,  task  conditions 
will  generally  be  too  broad  to  meet  the  requirements  for  test  conditions. 

You  mudc  refine  and  redefine  conditions  until  they  Insure  standardized 
tests  and  eliminate  variable  Influences  on  Individual  performance  In  the 
test  situation. 

b.  The  test  conditions  will  be  contained  In  detail  In  the  Test  Control 
Officers  Manual  for  Administration.  When  the  entire  HOC  Is  completed, 
conditions  for  an  Individual  scorable  unit  may  be  consolidated  Into  a 
station  with  other  scorable  units.  Nonetheless,  It  Is  necessary  to 
specify  conditions  for  each  scorable  unit  as  developed. 

Most  of  the  Information  required  for  conditions  should  already  exist 
In  the  notes  made  when  you  developed  process /product  measures.  This  step 
then  requires  only  assembling  those  notes  In  a consolidated,  logical 
format . 


5-23.  DETERMINE  ENVIRONMENT  CONDITIONS. 

a.  Environment  conditions  affect  outdoor  tasks  much  more  than  Indoor 
tasks  or  tasks  performed  under  cover.  Normally,  adverse  environmental 
conditions  will  be  the  primary  concern.  As  a minimum,  consider  the 
following  environmental  conditions: 

-Light  -Visibility  -Temperature 

-Precipitation  -Noise 

b.  Does  the  environment  affect  behavior?  Often  the  task  analysis 
will  state  that  the  task  Is  "performed  under  all  conditions."  However, 
If  one  soldier  does  the  task  In  the  rain  and  another  soldier  In  clement 
weather,  will  the  test  be  fair? 

RepMsber,  we  are  talking  about  test  conditions,  not  Job  conditions. 
When  stating  environment  requirements,  a general  statement  to  avoid 
adverse  conditions  usually  suffices,  for  example,  during  daylight  hours 
with  clement  weather  above  zero  Celsius. 
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c.  Naturally,  there  will  be  geographical  locations  where  envlron- 
■ental  conditions  cannot  be  aet.  Geographical  extremes  (Alaska  in 
the  %d.nter;  Canal  Zone  In  the  suaaer)  probably  will  have  to  be  considered 
as  special  conditions  for  test  adalnlstratlon.  Separate  Instructions 
will  be  addressed  only  to  the  TCO's  In  the  geographical  extreaes  Involved, 
At  this  point,  be  concerned  only  %flth  general  Instructions  affecting  the 
aajorlty  of  locations  In  which  this  test  will  be  administered. 


5-24.  DETERMINE  EQUIPMENT  CONDITIONS. 

a.  This  Is  one  of  the  most  Important  aspects  of  hands-on  testing. 
Consider  what  the  soldier  must  be  given  In  equipment  when  he  arrives  at 
the  station  for  testing.  It  will  not  suffice  to  have  Just  the  equipment 
"there;"  It  must  be  In  a specific  position. 

b.  Consider  first  what  equipment  Is  needed  lo  perform  the  task. 

List  all  Items  of  equipment  normally  used  on  the  Job  when  the  task  Is 
performed.  Include  manuals,  JPA,  or  other  references  ^ they  are  used 
on  the  Job. 

Include  all  tools  and  ancillary  equipment  and  materials  such  as 
cleaning  material  and  POL  products.  Consider  equipment  needed  by  the 
scorer  to  score  the  test  and  to  keep  the  station  operating  over  a period 
of  time. 


c.  Consider  all  aspects  of  the  equipment  Itself.  Specify  settings 
and  position  of  dials,  valves,  controls  and  switches.  Tell  the  scorer. 
If  necessary,  how  to  build-in  faults,  and  also  how  to  eliminate  unwanted 


d.  Specify  how  to  setup  equipment  for  the  first  examinee.  Also 
specify  what  to  do  In  preparing  equipment  for  the  second  examinee  so 
It  Is  laid  out  exactly  as  for  the  first  examinee.  Specify  layout  of 
materials  and  equipment  used  In  conjunction  with  the  major  Item. 


5-25.  DETERMINE  STATION  CONDITIONS. 

a.  Station  conditions  pertain  to  the  area  where  the  test  is  conducted! 
Station  cone  ‘'ions  may  be  Incidental  to  the  task,  like  space  to  work  and 
an  area  conducive  to  testing. 

Conditions,  however,  may  be  an  integral  part  of  the  test,,l.e.,  the 
task  can  be  performed  only  If  the  station  conditions  are  as  specified. 

For  example,  boresighting  a tank  gun  requires  a line  of  sight  of  1200 
meters. 
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b.  Station  requirements  depend  on  the  task  but  consider  as  a minimum 
the  following  requirements: 

-Indoor,  outdoor  'distance  requirements 

-Shop,  motor  park,  workbench  -Terrain,  vegetation 


5-26.  ARE  ALL  CONDITIONS  AFF^TING  BEHAVIOR  DETERMINED?  By  this  point 
most  requirements  to  standardize  test  conditions  should  have  been  met. 

Howver,  review  performance  measures  to  be  sure  that  conditions  pro- 
vide for  both  administration  of  the  test  and  peformance  of  the  task  by 
the  examinee.  If  a question  exists  as  to  the  possible  course  of  the 
examinee's  behavior  (or  what  he  will  do),  reatate  the  conditions. 


5-27.  SPECIFY  CONDITIONS  IN  iHE  T^SaipUMIlEIMML  EXAMINEE  INSTRUCTIONS. 

a.  Translate  condltlona  into  specific  Instructions  to  people  who 
conduct  the  test. 

b.  Describe  condition  of  site,  environment  and  equipment  for  the 
TCO/TSM  to  Implement.  Incorporate  those  statements  In  the  test- 
specific  Manual  for  Administration. 

c.  Include  specific  Instructions  to  the  scorer  for  setting  up  site, 
laying  out  equipment,  and  operating  station  In  the  Manual  for  Admlns- 
tratlon. 


5-28.  ARE  TEST  LIMITS  REFLECTED  IN  CONDITIONS? 

a.  All  conditions  establish  how  much  will  be  tested.  Review  start 
point  or  Initiating  cues  and  end  point  and  Insure  these  are  reflected 
In  the  conditions. 

For  example,  to  test  a soldier's  ability  to  adjust  a wheel  bearing, 
we  may  not  be  concerned  with  teatlng  the  soldier's  ability  to  Jack  up 
the  vehicle  and  remove  the  hub.  Therefore,  the  equipment  condition 
specifies  that  the  wheel  Is  raised  and  blocked  and  the  hub  removed. 

b.  If  limits  are  not  reflected  In  the  conditions,  you  may  need  to 
redefine  or  restate  conditions.  Insure  that  the  soldier  does  not  have 
to  do  more  or  less  than  you  want  to  test. 


i 


5-17 


I 


ARI  P-77-5 

5-29.  ARE  ANY  CONDITIONS  UNNECESSARILY  RESTRICTIVE? 

a.  Differentiate  between  what  Is  really  needed  to  Insure  standard- 
ized performance  and  what  Is  "nice  to  have." 

For  example,  shop  stalls  or  maintenance  bays  enhance  maintenance. tasks, 
but  they  are  not  required  for  all  maintenance  tasks.  Although  68°F  is 
a nice  indoor  working  temperatU'*e,  it  is  rarely  the  only  acceptable 
temperature. 

b.  Can  the  conditions  be  met  at  test  sites  in  the  field?  TDA  often 
have  more  available  equipment  and  better  terrain  than  do  field  units. 

An  example  is  specifying  the  use  of  stopwatches  which  are  unobtainable 
In  any  quantity  In  the  field.  While  stopwatches  are  required  for  time- 
constrained  tasks,  they  should  not  be  required  for  all  tasks. 

c.  Remember,  the  more  conditions  you  Impose  that  the  scorer  or  TCO 
cannot  meet,  the  more  likely  he  Is  to  Ignore  the  ones  that  are  necessary. 


Section  V.  PREPARE  EXAMINEE  INSTRUCTIONS 
5-30.  INTRODUCTION. 

a.  During  this  block,  you  will  decide  what  Information  is  going  to  be 
given  to  the  soldier  when  he  arrives  at  the  station  to  perform  the  task. 
The  soldier  must  quickly  and  concisely  comprehend  what  he  Is  to  do  and 
mentally  prepare  himself  for  doing  the  task. 

Remember,  the  scorer  In  the  field  will  not  be  allowed  to  answer  ques- 
tions or  expand  on  your  Instructions.  He  can  only  re-read  your  instruc- 
tions to  the  soldier  who  has  a question  or  doesn't  understand  what  he.  Is 
supposed  to  do. 

b.  Ability  to  retain  the  spoken  word  Is  limited  In  all  of  us.  Add  to 
this  the  pressure  that  will  exist  for  many  soldiers  in  a test  situation. 
Be  concise  and  direct.  Be  clear,  but  don't  "speak  down"  to  the  soldier. 

Finally,  remember  that  you  are  probably  writing  words  that  will  be 
spoken  or  recorded.  After  you  have  prepared  your  Instructions,  read  them 
aloud  to  a few  colleagues.  Do  they  know  what  you  want  them  to  do? 

Rework  the  Instructions  until  they  are  clear,  but  do  not  confuse  length 
of  Instructions  with  clarity. 
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5-31.  STATE  THE  JOB  SITUATION. 

a.  State  the  Job  role  the  aoldler  la  in  for  this  teat.  This  will 
noraally  be  hla  actual  Job.  Some  Job  situations  require  him  to  assume 
different  "mental  sets."  It  may  be  necessary  to  orient  the  soldier's 
thinking  to  a particular  piece  of  equipment,  a crew  position,  a function, 
a mission,  or  even  a task. 

Example:  "You  are  the  driver  of  an  M113,"  "You  are  conducting  a pris- 
oner count  at  a confinement  facility,"  "You  are  troubleshooting  an  air- 
craft electrical  system." 

b.  Not  all  tasks  need  a Job  situation,  e.g.,  masking;  M16  assembly; 
but,  unless  you  are  sure,  state  the  Job  situation. 

c.  Do  not  confuse  Job  situation  wl;h  unrealistic  role  playing.  To 
teat  splinting  a broken  arm,  tell  him:  "This  soldier  has  a broken  arm. 

It  will  be  some  time  before  you  can  get  medical  attention,  etc..." 

Do  not  Involve  the  soldier  with  unnecessarily  detailed  situations: 

"You  are  a member  of  a long-range  patrol  returning  from  a night  mission. 

A member  of  your  patrol  has  fallen  and  broken  his  arm.  You  must  move 
5 km  through  enemy  territory,  etc..." 


5-32.  DOES  SITUATION  UNDERCUE  OR  OVERCUE? 

a.  Does  It  provide  too  little  Information  (undercue)  to  start  the 
task?  Ask  yourself,  given  this  Information  what  will  be  my  first  step? 
If  not  clear,  give  more  Information. 

b.  Does  it  tell  the  soldier  what  he  Is  supposed  to  do  when  this  Is 
not  our  Intent?  Some  tasks  require  that  the  soldier  deduce  his  actions 
from  the  surroundings  or  conditions.  Are  you  telling  him  more  informa- 
tion than  he  would  have  at  this  point  on  the  Job?  If  so,  rephrase  the 
situation. 

For  example:  "You  are  setting  up  the  Claymore  mine.  You  have  not 
conducted  the  circuit  check  yet..."  overcues  the  soldier  by  telling 
him  what  he  Is  supposed  to  do. 


5-33.  STATE  WHAT  EXAMINEE  IS  TO  DO.  This  helps  to  direct  the  soldier's 
efforts  and  helps  to  establish  limits.  State  simply  what  It  Is  the 
soldier  Is  to  do.  For  example:  "You  are  to  set  up  a Claymore  mine  for 
comman4  detonation." 

It  may  be  necessary  to  Include  a stateswnt  on  how  far  the  soldier  Is 
supposed  to  go  (Halt)  If  the  task  Is  not  being  tested  to  Its  normal 
limits.  For  example:  "It  will  not  be  necessary  to  replace  the  engine 
cover;"  "Do  not  enter  the  control  number  In  the  log." 
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Orient  the  soldier  toward  the  first  step  in  the  task  in  a way  that 
approximates  the  situation  on  the  Job.  Avoid  the  situation  where  the 
soldier  could  have  several  correct  starts  unless  that  is  part  of  the 
task,  e.g.,  troubleshooting. 

b.  Do  not  undercue.  Generally  avoid  terms  like  "Take  appropriate 
action"  to  begin  the  task  unless  it  is  cleat  from  the  situation  what 
the  soldier  should  do. 

c.  Do  not  overcue.  Do  not  tell  the  soldier  how  he  will  perform 
the  task.  For  example:  "You  are  to  put  on  the  protective  mask  and 
clear  it." 


5-34.  STATE  TIME  AND  ACCURACY. 

a.  Always  give  the  time  limit. 

b.  Ehiphaslze  time-constrained  task  limits  by  including  the  time  limit 
in  the  statement  of  what  the  soldier  is  to  do.  Example:  "You  must  mask 
within  nine  seconds  from  when  I announce  'GAS'." 

c.  Give  the  examinee  the  standard.  Where  a range  of  performance  is 
acceptable  or  where  it  must  be  exact,  tell  him.  Example:  "Your  final 
charge  must  be  plus  or  minus  1/8." 

d.  Do  standards  overcue?  Do  not  allow  standards  to  overcue  In  the 
sense  of  revealing  information  on  how  to  perform  a step  or  task.  For 
example:  "After  you  return  to  the  aiming  point,  the  level  bubble  must 
must  be  centered." 


5-35.  STATE  ASSISTANCE  THAT  WILL  BE  PROVIDED. 

a.  If  the  task  allows  assistance,  state  exactly  what  that  assistance 
will  be.  For  example:  "I  will  position  the  aiming  stakes  as  you  direct 
me;"  "I  will  issue  the  fire  command  to  you." 

b.  Except  where  obvious  (for  example  in  masking)  state  that  there 
can  be  not  assistance  if  none  is  provided. 


5-36.  ARE  ALL  INSTRUCTIONS  NECESSARY  FOR  TASK  PERFORMANCE?  Examine  the 
complete  set  of  instructions.  Is  anything  in  the  Instruction  not  needed 
for  task  performance?  Are  terms  explained  that  need  no  explanation?  Are 
the  instructions  too  long? 

(Anything  over  four  or  five  medium-length  sentences,  or  over  15 
seconds  of  total  speaking  time  should  be  examined  carefully.)  Eliminate 
any  portion  of  the  instructions  not  essential  to  the  task. 
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5-37.  CAN  THE  EXAMINEE  UNDERSTAND  THE  INSTRUCTIONS? 


a.  This  Is  a Judgment.  la  the  terminology  clear?  Are  the  atate- 
menta  of  what  the  aoldler  muat  do  preclae? 

b.  Uae  terminology  the  aoldler  will  underatand.  Do  not  say  "pre- 
acrlbed  load  Hat"  If  the  aoldler  calla  It  "PLL."  Converaely,  do  not 
call  It  the  M203  If  the  aoldler  calls  It  a "grenade  launcher."  Uae  both 
nomenclature  and  common  names  If  there  la  doubt. 

c.  Consider  having  the  examinee  repeat  the  Instructions  back  to  the 
scorer  (not  verbatim  but  enough  to  Insure  they  are  understood).  At  the 
very  least,  always  ask  the  examinee,  "Do  you  understand  what  you  are 
required  to  do?"  This  gives  the  soldier  a chance  to  hear  the  Instruc- 
tions again. 

5-38.  PRINT  OR  RECORD  THE  INSTRUCTIONS. 

I 

a.  Print  the  Instructions  to  be  read  by  the  scorer  or  record  them  If 
you  have  an  audio  recording  capability.  If  part  of  the  Instructions  are 
given  to  the  examinee  for  reference  during  the  test,  these  must  be  i 

extracted  and  printed  on  a separate  reference  card.  | 

It  Is  generally  not  effective  to  have  the  examinee  try  to  read  the 
Instructions  at  the  same  time  as  they  are  being  read  to  him,  as  tne 
examinee  then  concentrates  on  neither. 


b.  As  part  of  the  Manual  for  Administration,  the  TDA  will  prepare 
a brief  set  of  Instructions  given  to  all  examinees  (probably  in  a group) 
before  they  start  the  test  component.  When  preparing  Instructions  for 
each  scorable  unit,  consider  %ihat  portions  could  be  placed  "up  front" 

In  the  general  Instructions. 


Try  to  present  only  instructions  that  apply  to  that  specific  scorable 
unit  (or  station).  However,  there  may  be  some  Instructions  that  need 
to  be  repeated  for  emphasis  at  all  or  most  of  the  stations. 


Section  VI.  PREPARE  SCORER  INSTRUCTIONS 


5-39.  INTRODUCTION.  To  Insure  a fair,  standardized  hands-on  test.  It  Is 
necessary  to  develop  specific  Instructions  for  the  scorer  for  each 
scorable  unit  In  the  component. 


In  developing  the  test,  you  may  discover  some  things  that  would  assist 
In  scoring  the  test.  In  short,  you  are  probably  now  well  qualified  to 
perform  as  a scorer  on  this  scorable  unit. 
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You  must  now  Impart  this  knowledge  of  how  to  score  the  test  to  an 
individual  who  has  not  had  the  benefit  of  your  detailed  analysis  of  the 
task.  The  scorer  must  be  more  than  a soldier  who  marks  PASS /FAIL  on 
a scoresheet.  He  Is  a test  station  manager. 

5-40.  SPECIFY  STATION  PREPARATION  REQUIREMENTS. 

a.  The  station  requirements  portion  of  the  test  conditions  provides 
Information  to  the  TCO  on  selecting  the  area  where  the  SU  station  will 
be  located.  It  Is  necessary  to  detail  specific  Instructions  to  the 
scorer  on  preparing  the  station  for  conduct  of  the  test. 

b.  For  Indoor  tasks,  you  may  want  to  Insure  the  station  Is  free  from 
visual  distractions  that  are  present  when  the  facility  Is  In  normal 
use.  For  outdoor  tasks,  you  may  Instruct  that  all  debris  be  removed, 
or  the  station  be  free  of  flammable  materials. 

c.  Station  preparation  requirements  must  not  be  beyond  capabilities 
of  one  or  two  Individuals  to  accomplish  In  a short  period  of  time. 


5-41.  SPECIFY  EQUIPMENT  SETUP  PROCEDURES. 

a.  State  In  step-by-step  procedures  precise  activities  that  scorer 
must  perform  before  each  examinee  Is  tested.  Include  methods  of 
controlling  all  variable  conditions  which  affect  standardization. 

b.  Specify  settings  of  dials,  knobs  or  adjustments;  position  of 
equipment  and  tools,  manuals  and  other  material  used  In  conjunction  with 
the  test. 

c.  Use  schematic  drawings  or  layout  diagrams  to  specify  the  setup  of 
equipment . 


5-42.  SPECIFY  MAINTENANCE  GUIDELINES. 

a.  Specify  maintenance  to  perform  at  regular  intervals  during  test 
administration. 

b.  Specify  maintenance  to  perform  before  eai  h examinee  Is  tested. 

c.  Specify  maintenance  required  under  certain  unpredictable 
conditions. 

d.  Specify  maintenance  required  to  prevent  cueing  subsequent 
examinees. 
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5-A3.  CAN  MORE  THAN  ONE  EXAMINEE  BE  TESTED  AT  A TIME? 

a.  When  large  numbers  of  soldiers  are  tested.  It  Is  desirable  that 
more  than  one  Individual  be  tested  at  a time  at  a station.  (NOTE:  This 
Is  possible  only  tfhen  product  tests  are  being  used.) 

b.  Examine  the  scorable  unit  to  determine  If  this  Is  possible.  If  It 
Is,  you  must  determine  how  many  a single  scorer  can  test  feasibly  at  a 
time.  (Normally,  this  will  not  exceed  four.)  In  testing  more  than  one 
examinee  at  once,  you  will  have  to  decide  how  to  position  them;  you  may 
also  have  to  change  examinee  Instructions,  or  provide  other  special 
Instructions  to  the  scorer. 

c.  Whether  group  testing  Is  feasible  or  not,  you  must  still  specify 
where  the  examlnee(8)  should  be  at  the  start  of  the  test,  when  and  how 
he  Is  to  be  provided  Information  during  the  test,  when  he  should  be 
allowed  to  see  the  equipment  layout,  and  how  to  handle  examinee  questions. 


5-A4.  DOES  TASK  PERFORMANCE  REQUIRE  SPECIAL  SCORER  PERFORMANCE? 

a.  If  the  scorer  or  assistant  scorer  Is  allowed  to  assist  the 
examinee  In  any  manner  during  the  test,  specify  extent  of  the  assistance. 

b.  If,  because  of  safety  or  other  considerations,  the  scorer  must 
Interfere  with  the  examinee's  performance,  tell  the  scorer  when  and  how 
to  Intervene  In  the  test. 


5-45.  SPECIFY  TIME  LIMITS.  State  specifically  when  the  scorer  must 
start  to  time  the  task  and  when  he  must  stop.  This  Is  especially  Impor- 
tant In  time-constrained  tasks.  The  points  must  be  easily  Identifiable. 


5-46.  ARE  SPECIAL  SCORING  TECHNIQUES  REQUIRED? 

a.  Specify  how  a scorer  must  measure  standards  If  this  Is  not  readily 
apparent.  For  example,  the  task  of  emplacing  the  M18A1  requires  the  scorer 
to  look  through  the  sighting  window  to  determine  alinement  with  the  aiming 
point . 

b.  Specify  when  and  how  to  use  scoring  aids  (templates,  models, 
photos)  If  they  are  provided. 

c.  Specify  checkout  procedures  on  equipment  to  eliminate  false  NO*^ 
caused  by  equipment  failures. 


5-47.  PRINT  SCORER  INSTRUCTIONS.  Scorer  Instructions  pertaining  to 
setting  up  equipment,  maintenance  and  special  situations  are  prepared 
In  a special  instruction  sheet.  Operation  of  the  station  must  be 
specified  In  the  Station  Annex  to  the  Manual  for  Administration. 
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CHAPTER  6 

VALIDATE  HANDS-ON  COMPONEOT 


Section  1.  TRY  OUT  HANDS-ON  COMPONENT  SU 


16-1«  INTRODUCTION*  This  tryout  Is  a systematic  way  to  have  experts  In 
^ the  MOS  review  the  HOC  to  be  sure  each  SU  Is  consistent  with  doctrine* 

You  must  test  experts  and  collect  their  opinions  of  the  SU  before  you 
submit  the  hands-on  component  to  ITED* 

6-2*  HAVE  TEST  DEVELOPERS  REVIEW  SU.  Before  you  begin  testing  experts, 

I have  two  or  three  people  who  have  helped  develop  an  SQT  review  the  score- 

I sheets  and  Instructions  for  the  SU*  The  test  developers  do  not  have  to 

be  qualified  In  the  MOS  covered  by  the  SQT* 


6-3*  IS  THE  SU  ACCEPTABLE?  Ask  each  test  developer  to  suggest  changes  \ 
In  the  SU*  Have  them  look  for  performance  measures  that  are  not  observ-  i 
able,  check  to  be  sure  the  standard  and  performance  measures  are  describee  | 
In  enough  detail  for  scorers  to  score  the  SU  reliably,  and  check  the  I 
clarity  and  completeness  of  the  instructions*  ^ 


6-4*  REVISE.  Weigh  the  suggestions  of  other  test  developers*  If  a 
suggestion  makes  sense,  make  the  recommended  changes*  Have  the  test 
developers  review  the  revised  SU* 

6-5*  SELECT  AT  LEAST  F-IVE  EXPERTS. 

a*  In  this  case  "experts"  are  people  who  yoi^  are  sure  can  do  the  task 
covered  by  an  SU*  They  do  not  have  to  be  In  the'  skill  level  fot  the  SQT. 
In  fact,  experts  will  usually  be  senior  NCO's  or  Instructors  in  the  MOS* 

An  officer  also  may  act  as  an  expert  for  this  tryout*  One  person  itey  be 
an  expert  for  several  tasks  covered  by  an  SQT*  Test  developers  should 
not  act  as  experts  for  this  part  of  the  tryout* 

b*  The  quality  of  this  tryout  depends  on  the  level  of  expertise  of 
the  soldiers  who  score  or  take  the  SU*  Do  %rtiatever  you  can  to  get  high- 
level  performers  for  the  tryout*  If  you  hsve  any  doubt  about >^he  qualifi- 
cations of  a potential  expert,  interview  his  peers  and  supervisor* 
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Ask  the  peers  and  supervisor  how  well  the  expected  expert  can  perform 
the  specific  task.  If  they  say  the  expected  expert  can  do  the  task  very 
well,  consider  Including  him  In  the  tryout.  If  they  say  the  expected 
expert  can  do  the  task  only  fairly  mil  or  worse,  do  not  Include  him  In 
the  tryout. 

c.  After  you  locate  several  people  you  expect  to  be  experts,  give 
them  a copy  of  the  task  analysis  data  for  the  SU  and  have  them  rate  their 
ability  to  perform  the  task  as  described  In  the  task  analysis  data.  Wheli 
you  collect  the  self-ratings,  emphasize  the  Importance  of  accurate  self- 
ratings. 

Be  sure  expected  experts  know  there  Is  no  benefit  to  them  or  to  the 
SQT  if  they  overrate  their  ability.  Use  one  of  these  questions  for  the 
self-ratings.  The  question  to  choose  depends  on  whether  the  task  Is 
performed  normally  under  supervision. 

(1)  How  well  can  you  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Not  very  well. 

2.  Fairly  well. 

3.  Very  mil. 

(2)  To  perform  the  complete  task,  how  much  technical  guidance 
would  you  require  from  someone  who  knows  how  to  perform  the 
task? 

0.  Not  familiar  with  the  task. 

1.  Lots  of  guidance. 

2.  Some  guidance. 

3.  No  guidance. 

For  this  tryout,  experts  are  people  »»ho  choose  No.  3,  "Very  well"  or  "No 
guidance." 


1 


6-6.  REVIEW  SCORESHEET  WITH  SCORERS.  Designate  two  of  the  experts  to 
act  as  scorers.  Go  over  the  scoresheet  with  the  scorers  to  be  sure  they 
understand  and  agree  with  the  performance  measures  and  standard. 


6-7.  ADMINISTER  SU  TO  EXPERTS.  Test  the  remaining  three  experts  one  at 
a tlm  on  the  SU.  Lay  out  the  equipment  and  set  up  the  station  the  way 
It  will  be  when  you  test  for  record.  Nave  one  scorer  read  the  Instruc- 
tions. Have  both  scorers  score  each  expert  who  Is  tested. 
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6-  8>  DO  SCORERS  AGREE  ON  RATINGS?  The  first  minimum  requirement  of  an 
SQT  hands-on  SU  Is  that  each  performance  measure  can  be  scored  consist- 
ently. If  the  scorers  disagree  on  any  performance  measure,  discuss 
the  disagreement  with  the  scorers  and  the  expert  who  was  tested. 

i 1 

f { The  disagreement  may  have  resulted  because  one  scorer  was  necessarily 

out  of  the  recommended  position  for  observing  the  procedure.  If  the 
I disagreement  resulted  because  of  Interpretation  of  the  scoresheet, 

> change  the  scoresheet  or  scorer  Instructions  to  eliminate  the  source 

I of  the  disagreement. 


6-  9.  SU  ACCEPTABLE? 

a.  The  second  minimum  requirement  of  an  SQT  hands-on  SU  Is  that 
people  who  take  the  SU  think  It  Is  a fair  measure  of  their  ability  to 
do  a critical  part  of  their  Job.  After  an  expert  Is  tested,  ask  him  or 
her  the  questions  In  Figure  6-2.  Ask  the  scorers  the  same  questions 
after  everyone  has  been  tested.  If  more  than  one  of  the  experts  or 
scorers  think  that  the  task  Is  non-crltlcal  or  that  the  test  Is  unfair 
or  Incomplete,  revise  the  SU. 

b.  Pay  special  attention  to  performance  measures  experts  fall.  Be 
sure  that  experts  who  fall  a performance  measure  agree  that  their 
performance  was  Incorrect.  If  they  do  not  agree  with  the  statement  of 
the  performance  measure,  check  the  accuracy  of  the  task  analysis  data. 

c.  Also  watch  for  "extra"  things  the  experts  do.  Discuss  those 
behaviors  with  the  expert  and  scorers  to  be  sure  they  are  optional 
behaviors.  If  the  experts  claim  they  are  essential  behaviors,  consider 
developing  performance  measures  for  them. 


6-10.  REVISE  THE  SU.  Your  primary  concern  during  this  tryout  Is 
that  the  SU  Is  consistent  with  doctrine.  If  two  scorers  disagree  about 
the  correctness  of  an  expert's  performance,  check  the  wording  of  the 
performance  measure  and  completeness  of  the  Instructions  to  the  scorers. 

If  experts  disagree  with  the  standard  for  any  performance  measure  or 
for  the  SU,  consider  changing  the  standard.  If  experts  think  the  task 
Is  not  critical  to  the  skill  level  addressed  by  ther  SQT,  convey  their  I 

reservations  to  the  person  who  selected  the  task  for  testing.  Have  the 
same  experts  review  any  revisions. 


6-11.  ADD  THE  SU  TO  THE  HOC.  A hands-on  SU  that  scorers  score  consist' 
ently  and  that  experts  consider  acceptable  meetd  first  requirements  for 
Inclusion  In  the  SQT. 

! I 


V" 
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6-12.  SUBMIT  HANDS-ON  CmPONENT  TO  ITED.  Submit  the  follot/lng  Items  to 
ITED: 

• The  scoresheet  for  each  SU. 

• Administrative  Instructions. 

•The  task  analysis  data  for  each  task. 

•The  SQT  notice  for  the  hands-on  component. 

Submit  the  HOC  at  the  same  time  you  submit  the  written  component. 


lOnce  the  HOC  has  been  approved  by 
I ITED  the  content  of  each  performance 
I measure  within  each  SU  is  fixed  and 
(cannot  be  changed  without  approval 
(from  ITED.  Performance  measures 
(found  to  be  unsatisfactory  during 
(validation  will  be  revised  to 
(improve  them,  but  not  to  the  extent 
(of  changing  content.  Unsatisfactory 
(performance  measures  cannot  be 
(deleted  without  approval. 
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EXPERT  AND  RATER  QUESTIONS  FOR  HANDS-ON  SCORABLE  UNITS 


Please  answer  the  following  questions  for  each  Scorable  Unit  Innedlatel^ 
after  It  Is  administered. 


1.  Is  the  task  covered  by  the  Scorable  Unit  a critical 
Job  requirement  at  the  skill  level? 

If  not,  why  not? 

2.  Does  the  Scorable  Unit  provide  a fair  measure  of 
job  requirements? 

If  not,  what  changes  are  required? 

a.  Add  Performance  Measures? 

Uhlch  ones? 

b.  Delete  Performance  Measures? 

Which  ones? 

c.  Modify  Performance  Measures? 

How? 

d.  Alter  standards? 

How? 

e.  Clarify  Instructions? 

How? 

f.  Change  test  site  or  equipment  requirements? 

How? 

3.  Is  the  Scorable  Unit  administratively  feasible? 

If  not,  what  changes  are  required? 

4.  Do  you  have  any  additional  comments  or  observations 
on  this  Scorable  Unit? 

If  yes.  Indicate  them  below. 


Figure  6-2.  Question  Sheet 
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Figure  6-3.  Velidite  Hends*On  SI* 
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Section  II.  VALIDATE  HANDS-ON  SU 


6-13.  INTRODUCTION.  This  part  of  the  validation  will  demonstrate 
whether  scorers  can  score  the  hands-on  SU  reliably  when  representative 
MOS  Incumbents  take  the  test.  It  will  also  give  Information  on  the 
acceptability  of  the  SU  to  the  representative  Incumbents. 


6-14.  SELECT  FIVE  SOLDIERS. 

a.  The  validation  sample  should  be  composed  of  soldiers  at  the  MOS 
skill  level  for  the  SQT.  If  several  skill  levels  will  take  the  SU,  try 
to  have  each  skill  level  represented  In  the  validation  sample. 

b.  Have  all  candidates  for  the  sample  rate  their  ovm  ability  to 
perform  tasks  covered  by  the  written  component.  V/hen  collecting  self- 
ratings,  emphasize  Importance  of  accuracy.  Be  sure  soldiers  know  there 
Is  no  benefit  to  them  or  to  the  SQT  If  they  overrate  their  ability.  Give 
each  soldier  a detailed  description  of  the  task  that  specifies  the  task 
conditions  and  standard.  Then,  ask  each  soldier  one  of  the  folloid.ng 
questions.  The  question  to  ask  depends  on  %d)ether  the  task  Is  performed 
normally  under  supervision. 

(1)  How  well  can  you  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Not  very  well. 

2.  Fairly  well. 

3.  Very  well. 

(2)  To  perform  the  complete  task,  how  much  technical  guidance 
would  you  require  from  someone  who  knows  how  to  perform  the 
task? 


0.  Not  familiar  with  the  task. 

1.  Lots  of  guidance. 

2.  Some  guidance. 

3.  No  guidance. 


c.  If  possible,  select  five  soldiers  who  choose  alternative  1 or  2. 
Have  as  few  soldiers  who  choose  alternative  3 as  possible.  Do  not  select 
anyone  who  chooses  alternative  0,  "Not  familiar  with  the  task." 


' A 
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6-15.  SELECT  FOUR  SCORERS. 

a.  Scorers  for  the  validation  should  meet  the  same  requirements  as 
scorers  for  the  actual  SQT.  They  should  hold  an  equal  or  higher  rank 
than  soldiers  In  the  sample  and  hold  the  MOS  the  SQT  addresses.  Ask 
potential  scorers  the  questions  In  paragraph  2.  Select  four  scorers  who 
choose  alternative  3. 

b.  Conduct  a rehearsal  for  the  scorers  by  having  them  act  as  examinees 

for  each  other.  Watch  each  scorer  administer  the  SU  at  least  once.  t] 


6-16.  ADMINISTER  SU  TO  SOLDIERS.  Test  soldiers  one  at  a time  and  have 
the  four  scorers  score  Independently  of  each  other.  Set  up  the  test 
conditions  as  much  as  possible  like  they  will  be  when  you  test  for  record. 
Measure  the  time  It  takes  each  soldier  to  perform  the  task. 


6-17.  DO  60Z  OF  SOLDIERS  ACCEPT? 

a.  Ask  soldiers  these  two  questions  as  soon  as  they  finish  the  SU: 

(1)  Do  you  think  this  task  Is  an  Important  part  of  your  MOS? 

(2)  Do  you  think  this  test  is  a fair  way  to  find  out  If  you  can 
do  this  task? 

b.  If  any  soldier  answers  "No"  to  either  question,  probe  to  find  the 
reason  for  this  answer.  If  the  test  Is  at  fault,  consider  changing  some 
characteristic  of  the  test  situation.  Each  question  must  be  answered 
"Yes"  by  three  soldiers  to  be  considered  acceptable.  If  more  than  two 
soldiers  do  not  accept  the  SU,  you  must  revise  It. 


1: 


6-18.  DO  75%  OF  SCORERS  ACCEPT? 

a.  After  all  soldiers  have  been  tested,  ask  the  scorers  these  two 
questions: 

(1)  Do  you  think  this  task  Is  an  laq>ortant  part  of  this  MOS? 

(2)  Do  you  think  this  test  Is  a fair  way  to  find  out  If  a soldier 
can  do  this  task? 


b.  If  any  scorer  answers  "No"  to  either  question,  probe  to  find  the 
reason  for  his  answer.  If  the  test  Is  at  fault,  consider  changing  some 


If  more  than  one  scorer  does  not 
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6-19.  COMPUTE  SCORING  AGREEMENT. 


a.  Collect  acoresheets  from  the  acorera.  Ihen  prepare  a auamary  of 
ecores  for  each  performance  meaaiire.  The  summary  should  show  the  score 
each  scorer  gave  each  soldier  on  the  performance  measure.  Here  is  an 
example  of  one  form  of  such  a summary: 

EXAMPLE  OF  A SATISFACTORY  PERFORMANCE  MEASURE 


Scorers 

1 

2 

3 

4 1 

1 

|1 

1 

1 

HI 

2 

|1 

1 

1 

HI 

I Soldiers 

3 

11 

0 

1 

HI 

4 

|1 

1 

1 

HI 

5 

|0 

0 

0 

on 

In  the  table  a "1"  means  the  scorer  gave  the  soldier  a "pass”  on  the 
performance  measure;  a "0"  means  the  scorer  gave  the  soldier  a "fall" 
on  the  performance  measure. 

b.  Next  prepare  a table  that  shows  how  many  times  each  pair  of  scorers 
agreed.  Here  la  a suggested  format  for  such  a table: 


An  "A"  means  the  scorers  agreed;  "D"  means  the  scorers  disagreed. 
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c.  Finally  divide  the  number  of  agreements  by  the  number  of  possi- 
ble agreements  (agreements  plus  disagreements).  In  this  case,  there  are 
24  agreements  and  30  possible  agreements;  therefore,  the  agreement  Is 
80Z. 

d.  Repeat  the  procedure  for  each  performance  measure  In  the  SU. 


6-20.  EIGHTY  PERCENT  (80Z)  SCORING  AGREEMENT.  The  minimum  requirement 
for  scoring  agreement  Is  80  percent.  This  means  that  a scorer  can 
disagree  with  the  other  three  scorers  no  more  than  twice.  A lower 
percentage  Indicates  a potentially  serious  level  of  unreliability. 


6-21.  IS  THE  SU  FEASIBLE? 

a.  Consider  three  questions  In  making  this  decision: 

•Is  the  time  limit  reasonable? 

•Will  the  length  of  this  SU  allow  the  Installation  with  the 
most  people  In  the  MOS  to  conduct  the  component  In  five 
days? 

•Are  the  test  conditions  essentially  the  same  for  each  soldier? 

b.  If  the  answer  to  any  of  these  questions  Is  "No,"  revise  the  SU 
to  Increase  Its  feasibility. 


6-22.  REVISE  THE  SU. 

a.  If  the  people  who  developed  the  SU  have  followed  the  development 
procedures,  minor  revisions  probably  will  make  the  SU  valid.  That  Is 
good,  because  there  Is  not  much  you  can  do  to  revise  a hands-on  test  that 
has  come  this  far  In  the  validation  procedure. 

b.  If  soldiers  or  scorers  do  not  accept  the  SU,  consider  changing 
the  test  conditions  or  changing  the  scope  of  the  task.  Soldiers  may  have 
resources  on  the  Job  not  available  at  the  test  site.  If  so,  make  those 
resources  available. 

Sometimes  part  of  the  task  will  be  done  by  someone  else.  Eliminate 
that  kind  of  irrelevant  requirement.  If  you  change  the  content,  coordi- 
nate with  ITED. 

c.  If  scoring  agreement  Is  too  low,  the  performance  measures  may 
not  be  stated  clearly.  Reword  such  performance  measures.  It  Is  also 
possible  that  one  of  four  scorers  must  stand  where  he  cannot  see  all  of 
the  performance. 
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Consider  that  possibility  before  making  major  changes  to  the  SU> 

If  the  disagreement  cannot  be  attributed  to  a superficial  weakness, 
contact  ITED  about  the  possibility  of  dropping  the  performance  measures 
from  the  SU« 

d>  If  you  revise  the  SU,  try  It  out  again.  You  may  use  the  same 
sample  of  soldiers  and  scorers  If  they  are  available.  If  the  same 
soldiers  and  scorers  are  not  available,  select  a new  sample  and  repeat 
the  data  collection  and  analysis. 


Section  III.  CONDUCT  FIELD  TRYOUT  OF  HOC 


6-23.  SELECT  APPROPRIATE  UNIT.  The  last  step  In  validating  a hands-on 
component  Is  to  determine  its  administrative  feasibility  In  a unit. 
FORSCOM,  Army  Reserve  or  National  Guard  units  may  be  selected  for  the 
site  of  the  field  cryout.  Coordinate  through  ITED  with  the  major  command 
to  select  a unit.  The  unit  selected  should  have  at  least  ten,  and 
preferably  up  to  50  soldiers  available  for  testing.  The  Intent  Is  to 
exercise  the  component  In  a realistic  environment. 

If  you  must  test  large  numbers  In  a brief  time  operationally,  have 
a large  sample  for  this  tryout.  If  you  must  test  only  a few  soldiers 
operationally,  you  may  have  fewer  than  10.  The  soldiers  must  be  In  the 
MOS  skill  level  that  will  take  the  hands-on  component. 


6-24.  SEND  TEST  MATERIALS  TO  TCO.  Send  the  TCO  who  will  conduct  the  test 
the  following  materials: 

• Twice  as  many  scoresheets  as  the  lumber  of  soldiers 
to  be  tested.  (The  scoresheet  will  serve  as  an  SQT 
Notice  to  the  soldiers  who  will  be  tested.) 

• A list  of  the  SU  to  be  tested  at  each  station. 

• Instructions  for  setting  up  the  test  site. 

• Instructions  for  setting  up  and  maintaining  each 
station. 

• Instructions  for  scoring  each  SU. 

• Requirements  to  be  a scorer  of  SU. 
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6-25.  OBSERVE  TESTING.  Send  a representative  from  the  TDA  to  observe 
the  testing. 
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6-26.  IS  SET-UP  CORRECT?  Be  sure  the  TCO  and  Che  scorers  have  follotred 
Instructions  for  setting  up  test  site  and  Individual  test  stations. 


6-27.  CHANGE  SET-UP.  If  the  setup  Is' different  than  Che  Instructions 
call  for,  learn  why.  If  the  new  setup  Is  an  Improveaent,  leave  It 
alone.  If  the  new  setup  might  make  test  conditions  different  than 
planned,  have  the  TCO  or  scorer  change  Che  setup  to  conform  to  require- 
ments. 


6-28.  REVISE  INSTRUCTIONS.  If  an  Incorrect  setup  Is  caused  by  unclear 
Instructions,  clarify  them.  If  a new  setup  Is  an  Improvement,  change 
Instructions  so  they  call  for  the  new  setup. 


6-29.  IS  HOC  ACCEPTABLE?  Have  each  soldier  and  scorer  fill  out  Question 
Sheet  2 (Figure  6-2).  Have  each  soldier  %/ho  Is  tested  fill  out  Question 
Sheet  4 (Figure  6-S).  If  more  than  20  percent  of  the  soldiers  and 
scorers  think  any  SU  Is  unfair,  review  conditions,  standard  and  perfor- 
mance measures  of  that  SU  for  possible  revision. 


6-30.  IS  HOC  FEASIBLE?  This  tryout  Is  your  only  chance  Co  find  out  for 
sure  how  many  scorers  and  how  much  equipment  will  be  required  Co  test  a 
given  number  of  soldiers  on  the  HOC.  Pay  special  attention  to  amount  of 
time  each  scorer  takes  to  administer  each  SU  at  the  station  to  one 
soldl'>r . 

Based  on  Chose  figures,  project  the  number  of  setups  of  each  station 
you  would  need  Co  test  the  largest  number  of  soldiers  In  the  MOS  at  one 
Installation  within  five  days.  Decide  whether  equipment  and  scorer 
requirements  are  reasonable. 


6-31.  REVISE  HOC.  If  requirements  are  not  feasible,  coordinate  with 
ITED  to  reduce  the  scope  of,  or  drop,  Che  troublesome  SU. 


6-32.  SUBMIT  HOC  TO  ITED.  Send  revised  scoresheets,  revised  Instruc- 
tions and  summaries  of  the  validation  and  field  tryout  Co  ITED. 
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QUESTION  SHEET  4 


In  MOS?**'*'**’^  ® y°"  ability  to  perform 


Test  1 


Test  2 


Test  3 


If  not,  please  explain  why. 


Figure  6-5.  Examinee  Questions  for  Hands-On  Scorable  Units 
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CHAPTER  7 


CONSTRUCT  WRITTEN  SCORABLE  UNIT 


OVERVIEW 


1.  OBJECTIVE.  The  objective  of  the  Written  Component  (WC)  Is  to  test 
the  soldier's  ability  to  perform  a task  by  evaluating  his  responses  to 
written  questions.  The  objective  should  not  be  thought  of  as  Just  a 
test  of  knowledge,  since  all  questions  should  be  oriented  toward  perform- 
ance. Knowledge  or  recall  should  be  no  more  than  required  when  the 
soldier  actually  performs  the  task. 

In  this  respect  the  written  component  of  the  SQT  differs  In  approach 
from  those  traditional  MOS  tests  that  relied  on  reading  and  recalling 
Information  that  often  was  not  essential  to  task  performance.  In  fact, 
the  best  way  for  a soldier  to  "study"  for  the  SQT  would  be  to  practice 
performing  the  task  correctly  - not  reading  about  It. 


2.  TASKS.  Tasks  for  which  written  scorable  units  are  to  be  developed 
will  fall  Into  three  categories: 

a.  Written  performance. 

b.  Performance-based. 

c.  Mixture  of  written  performance  and  performance-based. 


3.  WRITTEN  PERFORMANCE.  A written  performance  test  measures  the 
examinee's  ability  to  perform  an  actual  task  at  his  test  station  (desk). 
He  performs  whatever  steps  are  necessary  to  arrive  at  the  task  product, 
then  chooses  his  answer  from  real  world  alternatives  presented  In  the 
test  booklet. 

For  example,  a written  performance  SU  might  Involve  computing  the 
charge  for  a 4.2  mortar  round.  Given  certain  Information  and  Job 
relevant  "tools,"  the  soldier  actually  would  have  to  perform  the  task 
to  select  the  correct  answer. 


4.  PERFORMANCE-BASED.  A performance-based  test  measures  the  examinee's 
ability  to  answer  questions  about  how  he  would  perform  a task.  For 
example,  an  examinee  might  be  asked  to  Identify  before  operations  main- 
tenance checks  from  a larger  list  of  maintenance  checks. 
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Figure  7-1 . Develop  Written  Scorable  Unit  Overview 
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5.  MIXTURE  OF  WRITTEN  PERFORMANCE  AND  PERFORMANCE-BASED  TASKS.  This 
category  simply  combines  the  two  approaches.  Many  tasks  cannot  be 
tested  fully  using  a written  performance  approach. 

For  example,  given  the  task  "Navigate  Cross  Country  Using  a Map,"  an 
examinee  could  be  glvn  a mapsheet,  protractor,  and  a situation,  and  could 
actually  determine  coordinates,  measure  distance  and  convert  grld/mag- 
netlc  azimuths. 

The  examinee  could  then  be  asked  questions  about  measuring  pace  and 
using  the  compass.  We  still  have  not  tested  his  ability  to  move  cross- 
country, but  we  have  tested  his  ability  to  perform  several  of  the  steps, 
and  tested  his  knowledge  on  how  remaining  steps  should  be  accomplished. 

We  can  then  draw  the  conclusion  that,  barring  some  physical  Impairment, 
the  Individual  should  be  able  to  "Navigate  Cross-Country  Using  A Map." 


6.  To  determine  the  type  of  written  SU  you  will  develop,  refer  to 
the  flow  diagram  which  provides  an  overview  of  the  written  SU  procedure. 
By  taking  each  task  assigned  to  the  WC  and  follo%d.ng  the  procedure 
outlined  earlier,  you  will  be  able  to  assign  each  task  to  one  of  the 
three  categories. 


Section  I.  DETERMINE  WRITTEN  PERFORMANCE  OR  PERFORMANCE-BASED 


7-1.  INTRODUCTION. 


a.  Generally,  If  the  product  of  a task  can  be  measured  in  the  written 
component,  you  will  end  up  with  a written  performance  SU. 


Similarly,  If  the  product  of  an  entire  task  cannot  be  tested  In  the 
written  component,  but  the  product  of  one  or  more  of  the  task  elements 
can,  at  least  part  of  the  SU  will  be  written  performance. 


Where  none  of  the  task  elements  can  be  product-tested,  however,  you 
will  end  up  developing  a pure  performance-based  SU.  Depending  on  the 
type  of  task  you  are  working  with,  many  written  scorable  units  will  be  a 
mixture  of  written  performance  and  performance-based  Items. 


The  considerations  diagrammed  In  Figure  7-2  should  guide  your  decision 
as  to  whether  a whole  task  or  task  elements  can  be  tested  via  a written 
performance  Item,  or  whether  one  or  more  of  the  task  elements  must 
be  tested  via  performance-based  written  Items. 
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Figur.7.2.  D«ermir»  Written  PwfomMoc.  or  P.rform.n»B«ed 
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b.  A good  rule  of  thumb  to  use  in  distinguishing  between  written 
performance  and  performance-based  trrltten  items  is  to  ask  yourself; 
"Does  the  item  call  for  the  soldier  to  perform  the  task  or  task  element 
as  he  would  perform  it  on  the  Job,  or  does  the  item  call  for  him  to 
answer  questions  about  how  he  would  perform  it?" 

c.  The  following  explanation  should  help  you  answer  the  questions 
in  the  flow  chart. 


7-2.  CAN  THE  ENTIRE  TASK  BE  PERFORMED  AT  THE  SOLDIER'S  WC  STATION?  In 
essence,  this  is  asking  if  the  Job  task  can  be  enacted  on  the  written 
test,  given  the  limitation  of  work  space  (individual  work  space  for  the 
WC  is  assumed  to  be  equivalent  to  the  area  of  a standard  field  table). 

Where  equipment,  tools,  space,  terrain  and  so  forth  are  clearly  not 
manageable  at  the  written  test  station,  consider  simulation.  If,  through 
simulation,  you  still  cannot  enact  the  task  within  the  confines  of  the 
written  test  station,  reject  the  possibility  of  testing  task  performance. 
But  it  still  may  be  possible  to  test  performance  of  one  or  more  of  the 
separate  task  elements. 


7-3.  CAN  A TASK  ELEMENT  BE  PERFORMED  AT  WC  STATION?  This  question 
pertaining  to  an  individual  task  element  should  be  answered  in  the  same 
way  as  described  above  for  the  entire  task.  If  th>?  answer  is  "no", 
prepare  a performance-based  item  for  the  element.  If  the  answer  is 
"yes",  ask  yourself  the  next  question. 


7-A.  CAN  CORRECT  TASK  PRODUCT  BE  RECOGNIZED  BY  EXAMINEE  WITHOUT  ACTUALLY 
PERFORMING  TASK  OR  TASK  ELEMENT? 

a.  If  the  correct  product  of  the  task  or  task  element  can  be  recog- 
nized by  the  soldier  without  actually  performing  the  task  or  element, 
there  is  no  benefit  to  having  him  perform. 

Remember,  a requirement  of  the  written  component  is  that  the  soldier 
must  mark  his  answers  on  a mark -sense  answer  sheet.  Because  of  this 
constraint,  the  recorded  test  response  made  by  the  soldier  will  be 
limited  necessarily  to  a recognition  response. 

That  Is,  in  all  cases  he  will  come  up  with  an  answer,  compare  his 
answer  with  the  listed  alternatives,  and  select  the  alternative  that 
corresponds  most  closely  to  his  answer.  So,  if  without  actually 
performing,  he  can  generate  a correct  answer  for  comparison  with  the 
lis't  of  alternatives,  it  is  not  a performance  test. 


I 
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Disassembling  or  assembling  a piece  of  equipment  such  as  an  M16  rifle 
is  an  example.  Having  the  soldier  disassemble  the  weapon  at  the  written 
test  station  accomplishes  nothing,  since  he  Is  able  to  recognize  pictures 
or  verbal  descriptions  of  the  product  (the  weapon  In  some  number  of 
parts)  without  doing  It. 

On  the  other  hand,  even  a skilled  soldier  could  not  recognize  the  | 

correct  distance  In  kilometers  along  a route  on  a map  without  first  ! 

measuring  It.  . 

Therefore,  if  task  product  can  be  recognized  without  its  being  | 

produced  through  performance,  you  should  go  to  a performance-based  . r 

testing  mode.  If  the  task  product  cannot  be  recognized  without  some  ' 

performance  you  have  a written  performance  test.  j 

I 

b.  Some  examples  of  task  or  element  products  that  can  be  tested  In 
a written  performance  mode  are: 

• Calculating  the  distance  along  a route  on  a map. 

• Determining  range  to  target  using  the  mil  formula. 

• Adjusting  the  M16  sight  on  the  basis  of  a three-shot 

group  (number  of  clicks  elevation).  ! 

i 

• Determining  a soldier's  eligibility  for  overseas  levy  j 

(eligible  versus  Ineligible).  I 

• Determining  the  minimum  readiness  status  of  a piece  | 

of  equipment  (Red,  Amber,  Green). 

• Diagnosing  the  fault  In  a piece  of  equipment  (part 
A versus  part  B,  versus  ...). 

c.  If  a task  or  task  element  can  be  performed  at  a WC  station  and 
Che  correct  task  product  cannot  be  recognized  unless  the  task  Is  per- 
formed, develop  a written  performance  test. 


7-5.  CONTINUUM.  You  may  find  yourself  unable  to  distinguish  whether  a 
test  Item  is  a performance  or  is  performance-based.  Do  not  be  overly 
concerned  with  this.  While  examples  can  be  given  of  true  performance 
and  true  performance-based  questions,  not  all  test  items  will  fall  so 
neatly  Into  the  two  categories*  Think  of  the  distinction  as  being  along 
a continuum. 


Performance ???77??7?????? 


Performance- 

Based 
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At  either  end  the  distinction  is  easy*  But  somewhere  In  the  middle, 
individual  test  items  become  "fuzzy,"  being  neither  all  one  or  the  other. 
It  is  possible  to  have  a good  written  item  that  is  neither  truly  perform- 
ance or  performance-based.  But'  to  start  out  with  a clear  understanding 
of  the  differences  between  the  two  types  of  items  is  still  important. 


Section  II.  DEVELOP  WRITTEN  PERFORMANCE  ITEM 


7-6.  DEVELOP  TEST  SITUATION  IN  WHICH  EXAMINEE  PREPARES  PRODUCT.  Whether 
testing  an  element  or  an  entire  task,  begin  development  by  describing  the 
Job  situation.  This  statement  should  provide  all  information  necessary 
in  setting  the  stage  for  questions  you  are  going  to  ask.  While  necessary 
information  pertaining  to  task  conditions  and  the  Job  setting  must  be 
Included,  avoid  information  that  is  unnecessary  to  task  performance  since 
it  consumes  valuable  testing  time. 


|See  the  General  Situation"  in  the  land  navigation  example 
I (page  7-26,  Figure  7-5)  which  Is  also  an  example  of  an  SU 
(that  has  a mixture  of  performance  and  performance-based  items. 
(Also  Included  at  the  end  of  this  chapter  is  an  example  of  an 
|SU  consisting  solely  of  written  performance  items  (Example  2 
I of  Figure  7-5). 
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7-7.  ARE  JOB  STIMULUS  CONDITIONS  TRANSIENT? 

a.  Some  things  the  soldiers  must  observe,  listen  to,  or  otherwise 
sense  in  a Job  setting  may  be  present  for  a brief  time  only  in  that 
setting.  Examples: 

• A passing  aircraft  that  must  be  identified. 

• A briefly  exposed  tank  that  must  be  Judged  friendly  or  enemy. 

• An  incoming  radio  message  Co  be  recorded. 

You  must  recognize  when  such  transient  stimuli  are  part  of  the  isk 
conditions  since  it  will  make  a difference  in  how  you  present  the 
stimulus  in  the  test. 

b.  Use  slides,  videotape  recordings,  or  audio  recordings.  If  the 
stimulus  condition  is  transient,  you  must  attempt  to  control  its  presen- 
tation to  Che  examinee  by  use  of  slides,  videotape,  or  audio  recordings. 

Although  the  form  of  the  stimulus  will  be  a simulation  (a  picture  or 
recorded  sound),  you  can  limit  its  exposure  time  to  that  which  would 
occur  in  Che  Job  situation. 

c.  Use  narrative,  pictures,  documents  or  simulative  device.  If  a 
stimulus  condition  is  not  transient  but  relatively  constant,  for  the 
soldier  to  react  to  or  operate  on,  Chen  it  should  be  presented  on  the 
test  or  as  a supplement  to  it.  The  stimulus  can  be  presented  as  a 
narrative  description,  a picture  or  diagram,  an  actual  document,  or  a 
simulative  device.  Examples: 

• Distance  on  a map  - 25  kilometers. 

9 Range  to  a target  - 1500  meters. 

9 Eligibility  for  overseas  levy  - yes. 

• Readiness  status  of-  a piece  of  equipment  - RED. 


7-8.  PREPARE  QUESTIONS  OR  STEMS.  Once  you  have  specified  Che  product  of 
Che  task,  you  can  prepare  a clear  question  to  ask  or  statement  to  complete 
(stem).  In  testing  a Cask,  the  question  may  be  preceded  by  a further 
narrowing  of  the  general  test  situation  already  prepared;  that  Is,  a 
specific  situation.  The  specific  situation  and  question  should  make  It 
clear  to  the  soldier  Just  what  he  is  to  produce. 
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Examples: 

• "What  Is  the  ground  distance  from  point  A to  grid  coordinate 
EG  158858?" 

• "After  you  return  from  your  squad  leader  gives 

you  a mission  to  reconnoiter  the  bridge  site  at>«>  • 

Using  the  grid-magnetic  angle  shown,  what  is  the  magnetic 
azimuth  you  will  be  'using?" 

7-  9.  PREPARE  COMMON  ERRORS  AS  ALTERNATIVES.  Identify  incorrect  products 
that  often  result  in  real-world  performance  of  the  task.  These  will  be 
alternatives  to  the  correct  answer  in  the  multiple-choice  question.  No 
certain  number  of  alternatives  should  be  used  in  an  item. 

The  number  of  alternatives  is  determined  by  the  number  of  different 
errors  that  commonly  occur  in  the  real  world.  Do  not  simply  generate 
alternatives  mechanically,  but  check  to  make  sure  they  are  reasonable 
options. 

For  example,  in  measuring  the  distance  along  a route  on  a map,  the 
straight  line  distance  (Ignoring  curves  in  the  route)  would  be  one  common 
error  leading  to  an  incorrect  product. 

Using  the  scale  of  miles  rather  than  meters  in  converting  the  measure- 
ment to  meters  might  be  another  common  error  to  use  in  identifying  an 
Incorrect  answer  alternative. 


7-10.  ARE  THERE  MORE  THAN  NINE  COMMON  ERRORS?  Select  the  nine  most 
critical  errors.  The  SQT  answer  sheet  provides  for  no  more  than  10 
alternatives.  In  some  cases,  there  will  be  no  more  than  10  real  world 
options  (including  the  correct  alternative)  that  are  even  possible, 
much  less  plausible.  Ihese  options  might  be  equipment  readiness 
categories:  RED,  AMBER,  GREEN;  target  classification:  friendly  vs 
enemy. 

When  more  than  nine  common  errors  occur  on  the  Job,  reduce  them  on 
the  basis  either  or  how  frequently  each  occurs  in  the  real  world,  or  how 
critical  each  error  is.  For  example,  in  identifying  types  of  aircraft, 
there  may  be  more  than  10  possible  types.  Since  for  a particular  ques- 
tion such  as  "What  is  the  name  of  this  aircraft?"  you  can  list  no  more 
than  nine  alternatives,  you  might  select  the  nine  aircraft  most  often 
confused  with  the  one  being  presented.  Or,  you  could  select  the  nine 
most  critical,  in  the  sense  of  critical  consequences  resulting  from  a 
particular  mlsidentificatlon. 
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7-11.  ARE  TOOLS,  AIDS,  OR  REFERENCES  NEEDED?  MAKE  TOOLS  AVAILABLE. 

a.  If  tools.  Job  aids,  or  references  are  necessary  or  considered 
acceptable  In  performing  the  task,  they  should  be  made  available  on  the 
test.  You  should  consider  the  possibility  of  using  a simulation  of  the 
tool  or  aid  If  It  Is  not  feasible  to  provide  the  actual  one. 

If  being  able  to  locate  Information  In  a set  of  references  was  previously 
Identified  as  an  essential  element,  manuals  or  references  should  be 
provided. 

If  It  Is  not  feasible  to  provide  the  actual  references,  extracts  of  the 
material  should  be  provided  to  permit  an  evaluation  of  that  portion  of 
the  essential  element  dealing  with  the  Interpretation  and  use  of  the 
Information.  Example: 

• Protractor,  compass  for  map-using. 

: •TM  or  extracts  from  TM. 

I 

I b.  Some  cautions  are  necessary  when  considering  references.  It  Is 

seldom  possible  to  provide  each  examinee  with  a copy  of  a particular  TM  i 

or  other  reference.  Written  components  must  be  self-contained,  l.e., 
the  TCO  cannot  be  tasked  to  provide  references  from  his  resources  for  the 
WC. 

The  mo  t feasible  method  of  providing  references  Is  through  use  of 
extracts*  "o  to  30  pages  of  extracts  per  test  can  be  printed  by  ITED. 

If  more  / 'sn  30  pages  are  needed,  an  exception  to  policy  must  be  requested 
from  ITEi  « 1\  with  Justification  for  the  exception. 

j c.  For  this  reason  It  Is  probably  not  feasible  to  test  skills  having  j 

to  do  with  the  ability  to  locate  Information  In  a reference  In  the  WC. 

> 

If  necessary  to  test  this  ability.  It  should  be  done  In  the  HOC  by  ! 

developing  a special  SU  for  the  quasl-task  of  locating  Information  In  a ) 

TM  or  other  reference.  Limit  reference  extracts  In  the  WC  tc.  situations  j 

where  the  ability  to  use  the  Information  Is  the  critical  element.  i 

d.  You  must  rely  on  your  Judgment  and  that  of  subject  matter  experts  i 

In  deciding  whether  an  extract  Is  needed.  References  cannot  be  provided 
for  every  SU,  nor  Is  a reference  needed  Just  because  It  exists.  You  must 
consider  the  Job  requirements. 

I For  example,  the  procedure  for  clearing  a misfired  round  Is  contained 

In  the  IM.  Yet  the  soldier  does  not  refer  to  the  TM  when  a misfire  occurs. 

You  must  decide  what  material  the  soldier  In  the  MOS  and  skill  level  Is 
expected  to  Internalize. 


I 


I 
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Conversely,  do  not  discard  the  Idea  of  a reference  Just  because  some 
soldiers  In  the  MOS  (usually  "experts")  can  perform  the  task  without  use 
of  a reference.  This  could  lead  to  claims  of  "foul"  In  the  part  of  less- 
skilled  examinees  who  felt  discriminated  against  by  lack  of  references 
which  are  authorized  and  routinely  used  (by  them,  anyway)  on  the  job. 

Some  references,  published  by  commercial  Industry, 
developers,  or  even  Individual  authors,  are 
copyrighted.  AR  310-1  specifies  the  procedure 
for  dealing  with  copyrighted  material  and  how 
to  obtain  clearance  for  use.  If  copyrighted 
material  Is  used,  the  TDA  is  responsible  for 
obtaining  consent  In  writing.  A copy  of  the 
consent  should  be  Included  with  the  material 
when  It  Is  submitted  to  ITED. 

7-12.  ARE  ALL  KEY  ELEMENTS  COVERED?  In  Chapter  3 you  Identified  key 
elements  (those  steps  the  soldier  is  most  likely  to  fail  or  steps  that 
have  serious  consequences  of  failure).  Now,  be  sure  that  all  Identified 
key  elements  are  covered  In  the  written  scorable  unit. 

Other  steps  or  elements  may  be  covered  also,  but  there  must  be  at 
least  one  test  Item  for  each  key  element.  If  any  are  not  covered,  go 
through  this  development  procedure  for  each  remaining  element.  It  may 
be  necessary  to  use  the  performance-based  model  in  Section  III  to  Insure 
all  key  elements  have  been  covered. 


7-13.  IS  TEST  LANGUAGE  SAME  AS  JOB  LANGUAGE?  Be  certain  that  the  test 
Items  and  test  situation  communicate  to  the  examinee.  In  more  technical 
areas,  a tendency  Is  to  try  to  simplify  technical  Items  during  develop- 
ment of  items.  When  done,  however,  the  Item  may  lose  Its  Intended 
meaning. 

Technical  terms  used  on  the  Job  by  the  examinee  have  very  precise 
meanings.  If  used  routinely  on  the  job,  the  terms  probably  should  not 
be  changed  or  "simplified"  for  the  test.  All  test  materials  should  be 
written  In  the  normal  Job  language  and  reviewed  and  revised  until  they 
pass  this  checkpoint. 


7-14.  ARE  ITEMS  INDEPENDENT?  This  decision  point  Is  Included  to  Insure 
that  the  stem  for  one  Item  does  not  give  away  the  answer  to  another  Item. 
An  examinee  tdio  reads  the  stem  to  an  Item  should  not'  be  able  to  work 
backwards  and  correct  a previous  answer. 

Check  to  make  sure  you  have  not  answered  any  Items  In  the  test  situa- 
tion or  the  Item  stems.  If  you  cannot  reword  or  restructure  a cueing 
Item,  delete  the  least  Important  Item  that  provides  cueing. 


7-12 


ARI  P-77-5 


7-15.  PUT  ITEMS  IN  OPERATIONAL  SEQUENCE.  The  final  action  of  this 
process  is  to  place  items  in  a scorable  unit  in  operational  sequence. 

Tn  developing  this  sequence,  list  items  in  the  order  that  they  normally 
would  occur  on  the  job. 


Section  III.  DEVELOP  PERFORMANCE-bASED 
WRITTEN  SCORABLE  UNIT 


7-16.  INTRODUCTION. 

a.  The  performance-based  written  SU  likely  may  be  the  most  difficult 
to  develop.  Yet,  because  of  the  limitations  on  HOC  and  the  relatively 
limited  number  of  items  that  can  be  developed  in  the  written  performance 
mode,  it  is  the  type  of  SU  that  will  be  used  most  frequently  for  SQT. 

Development  is  difficult  because  we  are  still  trying  to  test  perfor- 
mance, not  abstract  knowledge  or  recall.  Since  the  test  is  performance- 
based,  direct  your  attention  to  %diat  happens  when  the  soldier  performs 
the  task.  Specifically,  focus  on  the  reasons  that  soldiers  frequently 
cannot  perform  the  task. 

b.  As  Illustrated  in  the  flow  diagram,  five  main  categories  of 
reasons  show  why  a soldier  may  fail  to  perform  a task.  Some  tasks  will 
fit  more  than  one  of  the  categories. 

When  going  through  the  flow  diagram,  go  through  each  of  the  five  top 
decision  diamonds  before  going  through  the  method  below  each.  This  is 
to  insure  that  all  categories  have  been  considered  in  selecting  the  most 
appropriate  causes  of  failure. 

7-17.  WHY  IS  PROFICIENCY  LOW?  Each  element  or  step  in  the  task  must  be 
considered. 

The  developer  should,  utilize  his  cwn  knowledge  of  soldiers  in  the 
skill  level,  available  subject  matter  experts,  plus  any  Information  he 
has  gathered  during  the  task  selection  process  to  determine  most  likely 
reasons  a soldier  cannot  perform  the  step. 

Although  no  formal  evaluation  of  the  step  is  required,  use  the  best 
resources  you  have  available  in  making  this  decision.  Take  each  element 
through  the  following  five  decision  points.  However,  not  all  elements 
need  be  included  in  the  test.  Some  elements  simply  pose  no  performance 
problems  for  soldiers  and  these  should  not  be  included  in  testing. 
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7-18.  DO  MOT  KNOW  WHERE  TO  PERFORM. 

The  failure  to  parfom  aona  taaka  nay  ba  due  to  the  Inability  to 
locate  certain  objccta.  For  exanplc,  a vehicle  driver  may  fall  to  pcrfoni 
checka  on  preventive  maintenance  on  the  component  of  a vehicle  becauae 
he  doea  not  know  where  that  component  ‘la  located. 

A mechanic  nay  fall  to  aero  a multimeter  becauae  he  does  not  know  the 
location  of  the  correct  adj^atnent  screw.  If  location  la  the  problem, 
follow  the  procedure  below.  (If  location  Is  not  the  problem,  go  to 
Paragraph  7-19.) 

a.  Identify  the  Correct  Location.  This  la  simply  deciding  what  the 
right  answer  la.  It  nay  be  necessary  actually  to  view  the  equipment  If 
the  element  is  equipment-oriented.  Technical  manuals  nay  not  have  a 
clear  plctura  or  a clear  description.  Be  alert  for  TMs  which  nay  ba 

In  error. 

b.  Can  Photoarapha  Show  Item  and  Surroundlnaa?  It  may  be  possible 
to  show  locations  by  the  use  of  photographs.  In  fact,  clear  photoa 
approximate  the  Job  presentation  closer  than  any  other  two-dimensional 
means.  If  photoa  are  not  practical,  consider  other  visual  means  such 

as  drawings  or  schematics.  Insure  that  the  photo  or  drawing  Is  properly 
keyed . 


c.  Produce  Description  of  Locetlon.  If  surroundings  of  the  component 
cannot  bf>  Illustrated  clearly  and  accurately  In  a picture  or  visual 
repreocatatlon,  then  location  of  the  component  should  be  described  as 
cltirly  as  possible. 

d.  Write  Questlon(s)  for  Location.  Develop  one  or  more  questions 
which  will  examine  thoroughly  reasons  for  failure  to  locate  the 
component.  For  example,  with  a keyed  picture  of  a multimeter,  the 
question  might  be: 

"Which  letter  In  Flture  represents  the  location  of  the  aero 
adjusting  screw?"  Or  If  no  picture,  the  question  might  be:  "Which  of 
the  following  best  describes  the  locetlon  of  the  aero  adjusting  scraw 
on  the  multimeter?" 

e.  Select  Real  World  Alternatives.  Consider  the  likely  options 
which  you  would  find  on  the  job.  Select  from  or  use  these  options  (no 
more  than  nine  since  there  are  only  ten  responses  per  question)  as 
alternatives  to  the  correct  answer. 

The  types  of  altsrnatlves  selectel  will  vary  depending  on  whether  the 
Item  uses  visual  or  verbal  lnformatl>n.  Fbr  a given  Item,  the  nuid>er 
of  alternatives  selected  Is  not  Important.  It  Is  Important,  however, 
that  the  alternatives  be  reasonable  and  realistic  In  terms  of  errors  that 
commonly  occur  In  the  real  world. 
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7-19.  DON'T  KNOW  WHEN  TO  PERFORM  STEP.  The  next  mejor  reason  for  fail- 
ure to  perforin  a task  la  that  the  soldier  may  not  know  when  in  a sequence 
of  activities  the  step  should  be  performed.  For  example.  It  Is  physi- 
cally possible  to  remove  the  hand  guards  on  the  M16  rifle  at  almost  any 
time  during  assembly. 

However,  there  Is  a point  In  the  sequence  %dien  this  step  should  be 
performed  If  the  soldier  is  to  avoid  damaging  the  weapon.  If  the  problem 
Is  determined  to  be  one  of  when  or  sequence,  use  the  following  procedure. 
(If  not  a problem  of  when,  go  to  paragraph  7-20.) 

a.  Identify  Correct  Step/Sequence.  This  means  Identifying  the  correct 
point  In  the  sequence  for  performing  the  step. 

b.  Write  Question  for  Sequence.  Develop  one  or  more  questions  that 


will  thoroughly  examine  the  reasons  for  falling  to  perform  this  step  In 
sequence.  The  question  might  be  phrased  this  way: 

"Which  of  the  following  steps  must  be  performed  before  ...?"  or, 
"Which  of  the  following  steps  must  not  be  performed  until  ...?"  If  the 
problem  Is  one  of  soldiers  performing  the  step,  but  at  the  wrong  point 
In  the  task,  the  step  may  be  given  either  In  the  question  or  as  one  of 
the  alternatives. 

But  if  the  problem  Is  one  of  soldiers  leaving  the  step  out.  It  would 
be  better  to  offer  It  as  an  alternative  while  referring  to  a preceding 
(or  succeeding)  step  to  key  the  sequence  In  the  question. 


c.  Select  Real  World  Alternatives.  Identify  reasonable  alternatives 
that  the  soldier  would  find  on  the  job.  Select  alternatives  with  empha- 
sis on  those  variations  which  Include  adjacent  steps.  Remember  that 
nine  alternatives  (plus  the  correct  answer)  are  the  limit.  Only 
Include  the  likely  variations  If  there  are  more  than  nine. 


7-20.  DON'T  KNOW  WHAT  THE  PRODUCT  IS. 

Another  reason  for  failure  to  perform  a task  correctly  Is  that  the 
Individual  may  not  know  what  the  end  result  of  the  element  or  step 
should  look  like. 

For  example,  a clerk  cannot  type  a correct  address  label  unless  he 
knows  how  a proper  address  Is  formatted.  A gunner  cannot  obtain  a 
proper  sight  picture  unless  he  recognizes  the  sight  sllnemcnt  when  he 
sees  It.  If  this  is  the  cause  of  soldiers'  failure,  use  the  following 
procedures.  (If  not  the  cause,  go  to  paragraph  7-21.) 

a.  Identify  Correct  Product.  The  first  step  Is  to  Identify  correctly 
what  Is  the  correct  outcome,  result,  or  product  of  the  step. 
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b.  Ig  Ability  to  "Read”  Product  Needed?  In  determining  whether  the 
correct  product  has  been  obtained,  an  Individual  uay  be  required  to  Judge 
or  interpret  diagrams,  meter  readings,  aerial  photographs,  charts, 
position  settings,  or  other  similar  visual  representations. 

If  this  kind  of  thing  Is  used  In  determining  whether  or  not  the  step  has 
been  completed,  then  pictures  or  visual  representations  should  be  used 
In  the  test  Items. 

c.  Prepare  Description  of  Product.  If  the  product  cannot  be  repres- 
ented by  some  visual  means.  It  Is  necessary  to  write  out  a descrip- 
tion of  the  product.  Insure  that  the  description  contains  the  discrimi- 
nating cues  that  the  soldier  mentally  uses  on  the  Job. 

For  example,  "needle  reading  Is  In  the  green  area"  or  "the  dial  Is 
rotated  until  the  detent  Is  engaged." 

d.  Write  Question (s)  for  Product.  Develop  one  or  more  questions 
which  will  examine  thoroughly  the  reasons  for  falling  to  produce  (Iden- 
tify) the  correct  product.  These  questions  should  center  around  the 
scope  and  basic  content  of  tdtat  the  correct  product  Is. 

For  example,  "Which  of  the  following  pictures  shows  the  correct  sight 
allnement  for  the  M16?"  or,  "When  conducting  an  operational  check  of  the 
field  telephone,  the  LOUD  switch  should  be  In  which  of  the  following 
positions?" 

e.  Select  Real  World  Alternatives.  Identify  all  of  the  reasonable 
variations  and  errors  expected  to  occur  on  the  Job.  Select  alternatives 
from  these  common  errors  which  would  lead  to  failure  to  produce/identify 
the  correct  product. 

Alternatives  selected  should  reflect  the  more  critical  errors  If  more 
than  nine  are  Identified. 


7-21.  DON'T  KNOW  HOW  TO  PERFORM  PROCEDURE. 

The  final  consideration  for  why  soldiers  cannot  perform  has  to  do  with 
procedure.  Soldiers  may  know  where  and  when  to  perform  a step,  and  what 
the  correct  result  Is,  but  they  may  not  know  how  to  produce  that  result  - 
that  Is,  how  to  execute  the  step. 

Procedural  failures  may  result  because  the  soldier  simply  does  not 
know  what  the  procedure  Is  or  because  he  forgets  It. 
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For  cxasplc,  In  preparing  Che  light  antitank  veapon  (LAW)  for  firing, 
ore  step  le  to  extend  Che  launcher  tube*  The  aoldler  m*y  know  that  the 
tube  Euat  be  extended  prior  to  firing  and  where  the  Cube  is  located  and 
what  the  LAW  would  look  like  with  the  tube  extended*  But  unleaa  he  knows 
how  to  extend  the  tube,  he  cannot  perform  the  task*  If  your  element 
falls  within  procedural  reasons  for  failure,  follow  the  steps  below* 

a*  Identify  Correct  Procedure*  As  with  all  reasons  for  failure  the 
first  step  is  to  Identify  the  correct  way  to  perform  the  element* 

b*  Write  Questions  for  Procedure*  Develop  one  or  more  questions 
which  will  examine  the  reasons  for  failing  to  perform  the  procedure 
correctly*  Several  approaches  are  possible*  The  soldier  may  be  asked 
to  select  correct  steps  from  several  presented,  to  place  steps  in 
sequence  or  to  Judge  a series  of  photographs  as  to  their  correctness* 

c*  Select  Real  World  Alternativec*  Select  the  nine  (or  less) 
responses  as  altarnatlvas  that  reflect  the  most  likely  incorrect  actions 
that  the  soldier  would  take  on  the  Job* 


7-22*  IS  THERE  ONE  ITEM  FOR  EACH  RET  ELEMENT?  Recall  the  earlier  work 
(Chapter  3)  in  selecting  key  elements*  Now  insure  these  key  elements 
are  covered  by  test  items*  You  may  decide  to  test  elements  not  design- 
ated as  key  elements  as  wall,  but  the  key  elements  must  be  covered* 

Each  key  element  should  be  evaluated  by  at  least  one  item  to  provide 
a comprehensive  examination*  In  some  cases  it  may  not  be  realistic  t"' 
develop  more  than  one  item  for  each  key  element* 

Examine  test  items  for  each  key  element  and  determine  if  critical 
aspects  of  the  element  have  been  omitted  in  terms  of  location,  sequence, 
product,  or  procedure*  Develop  additional  test  items  as  required  to 
provide  a comprehensive  evaluation  of  each  key  element* 


7-23*  ESTABLISH  A SCORING  STANDARD* 

a*  Once  you  have  developed  a Written  SU,  you  must  set  a passing  scora* 
This , involves  deciding  the  number  of  items  the  examinee  must  answer 
correctly  in  order  to  pass  the  SU*  In  most  cases  this  is  all  you  will 
have  to  do* 

Sometimes,  howsver,  your  work  will  be  complicated  by  an  item  in  the 
SU  having  more  than  one  correct  answer*  Then  you  will  have  to  decide  on 
a passing  score  for  the  Item  also*  The  following  offers  guidance  on 
determining  a passing  scora  for  an  SU  and  for  aa  item  with  multiple 
correct  answers* 
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b.  P«»alnt  Score  for  Scorabl*  Dnlt. 

(1)  Establlahlng  a paaalng  acora  for  an  SO  la  not  aa  atralght forward 
aa  we  sight  think*  One  can  argue  convincingly  that,  ainee  we  are  dealing 
with  perforaance-oriented  criterion-referenced  teating,  the  aoldier  suat 
paaa  every  item  in  order  to  paaa  the  SO.  And  in  theory  that  ia  true* 

There  are  however  extenuating  circuaatancea  - particularly  when 
written  teata  of  taak  perforaance  are  uaed*  Horde  aoaetiaea  get  in  the 
way  of  a aoldier  trying  to  coaaunicate  hia  or  her  knowledge  of  taak 
perforaance* 

Differencea  in  ability  to  read  and  underatand  the  printed  word  or  to 
focua  aental  concentration  on  words  for  a prolonged  tiae  aay  cause  ezaai- 
nees  to  aisread  or  aiainterpret  occasionally  soae  parts  of  the  written 
test,  no  aatter  how  carefully  the  developer  worded  the  questions* 

Since  these  lapses  should  not  necessarily  be  Interpreted  as  a lack 
of  knowledge  about  how  to  perfora  the  task,  we  should  allow  .the  exaainee 
soae  saall  aargln  of  error  in  the  Written  Coaponent* 

Research  has  shown,  in  fact,  that  written  test  scores  are  truer 
(reflect  ability  actually  to  perfora)  when  soae  aargin  of  error  is 
allowed. 

(2)  Recoaaended  passing  scores  for  SU'a  of  various  sizes  are  given 
below.  Rote  that  the  exaainee  is  not  allowed  an  Incorrect  answer  for 
SU's  with  4 or  fewer  Iteas;  but  is  allowed  one  aiss  for  an  SU  with  5 

to  7 iteas,  and  two  aisses  for  one  with  8 to- 10  iteas.  Using  these 
standards  for  scoring  SU's  should  Increase  the  validity  and  perceived 
fairness  of  your  Written  Coaponent* 

Nuaber  of  Iteas  Exaainee 
Must  Answer  Correct  to  Pass  an  SU 
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c.  Passing  Score  for  Item. 

(1)  For  reasons  similar  to  those  Just  given,  we  should  consider  also 
allowing  the  examinee  some  margin  of  error  on  Items  that  have  more  than 
one  correct  answer.  Such  Items  tap  several  knowledge  elements  at  once, 
elements  that  would  be  covered  separately  In  a more  conventional  multiple- 
choice  test. 

But  since  with  SQT  questions,  we  are  striving  to  maintain  task 
Integrity,  you  may  develop  Items  that  call  for  more  than  one  correct 
answer.  Those  Items  tend  to  be  more  difficult  for  examinees,  especially 
when  they  are  not  told  how  many  alternatives  to  mark. 

(2)  The  conventional  four-alternative  multiple-choice  item  has  one 
correct  answer.  The  examinee,  even  when  In  doubt,  marks  but  one  alter- 
native, knowing  that  he  will  surely  fall  the  Item  if  he  marks  more  than 
one  or  none  at  all. 

In  contrast,  an  examinee  responding  to  an  Item  with  multiple  correct 
irdwera.  If  at  all  uncertain  about  one  of  the  alternatives,  must  decide  i 

how  many  to  mark  while  deciding  %ihlch  ones  are  correct;  that  lS|  not 
being  told  the  exact  number  to  mark  makes  It  difficult  to  know  whether 
some  questionable  alternative  should  be  Included. 

For  this  reason,  Items  with  multiple  correct  responses  require  more 
precise  knowledge  of  the  subject  matter,  and  the  examinee  should  be 
allowed  some  small,  margin  of  error. 

(3)  Below  la  a table  of  recommended  passing  scores  for  Items  %rlth 

multiple  correct  answers.  Notice  that  to  make  the  table  complete,  we 
have  Included  the  case  of  one  correct  response.  More  Importantly,  notice 
that  the  passing  scores  shown  In  the  body  of  the  table  are  based  on  all  . 

alternatives  being  scored.  | 

t • 

The  maximum  error  allowed  tends  to  be  where  about  half  of  the  alter- 
natives are  keyed  as  correct.  In  no  case,  however,  Is  more  than  one 
error  each  of  omission  and  commission  allowed. 


I 
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Recommended  Passing  Scores*  For 
Items  With  Multiple  Correct  Answers 


1 

1 

Number 

of  Item  Alternatives 

4 

5 

6 

7 

8 

9 

10 

1 

4 

5 

6 

7 

8 

9 

10 

2 

3 

4 

5 

6 

7 

8 

9 

3 

4 

4 

5 

5 

6 

7 

8 

Number 

of 

4 

5 

5 

5 

6 

7 

8 

Correct 

Alternatives 

5 

6 

6 

6 

7 

8 

6 

7 

7 

7 

8 

7 

8 

8 

8 

8 

9 

9 

9 

10 

The  passing  scores  shown  in  the  table  are  based  on  all  alternatlvest 
l*e«,  both  narked  and  unmarked  alternatives  scored. 


(4)  To  use  the  table,  find  the  column  representing  the  number  of 
alternatives  in  your  test  Item,  then  read  down  that  column  to  the  row 
representing  the  number  of  correct  alternatives. 

The  number  at  the  intersection  of  that  row  and  column  Is  the  minimum 
number  of  Item  alternatives  that  must  be  correctly  marked  (and  left 
unmarked)  for  the  examinee  to  pass  the  Item. 

For  example,  suppose  you  have  an  Item  with  eight  alternatives,  four 
of  which  are  correct.  The  recommended  paslng  score  Is  six.  A soldier 
who  marked  two  or  three  of  the  four  correctly  and  did  not  nark  any  of 
the  Incorrect  alternatives,  would  pass. 

If  he  marked  all  four  correct  alternatives,  plus  no  more  than  two 
of  the  Incorrect  ones,  he  would  also  pass.  Or  If  he  failed  to  mark  one 
of  the  correct  ones  but  marked  no  more  than  one  of  the  Incorrect,  again 
he  would  pass. 
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(5)  If  these  passing  standards  sees  too  lenient » reaeid>er  that  In  no 
Instance  do  they  allow  aore  than  one  coablned  error  of  onlsslon  and 
coaalsslon. 


7-24.  IS  THE  TEST  LANGUAGE  THE  SAME  AS  THE  JOB  LANGUAGE?  Ton  nust 
Insure  that  the  test  Items  and  test  situation  conaunlcate  to  the  examinee. 
In  some  of  the  more  technical  areas,  there  Is  a tendency  to  try  to 
simplify  technical  terms  during  the  development  of  Items  ostensibly  In 
the  Interest  of  communication. 

When  this  Is  done,  however,  the  Item  may  lose  Its  Intended  meaning. 
Technical  terms  are  used  on  the  Job  because  they  have  very  precise 
meanings.  All  test  materials  ^ould  be  written  In  the  normal  Job  lan- 
guage and  reviewed  and  revised  until  they  conform  completely  with  the 
current  Job  language. 


7-25.  ARE  ITEMS  INDEPENDENT?  This  decision  point  Is  Included  to  Insure 
that  the  answer  to  one  question  does  not  Influence  or  Is  not  used  In  the 
selection  of  the  answer  to  another  question.  This  Is  not  to  say  that 
the  Items  cannot  be  related. 

The  test  situation  for  an  SU  normally  will  provide  a relationship 
between  all  the  Items  In  that  unit,  but  the  answer  to  one  question 
should  be  completely  Independent  of  the  selection  of  the  answer  to 
another  question. 

Items  should  be  Independent  from  the  standpoint  of  cueing.  The  stem 
of  an  Item  should  not  cue  the  answer  to  a previous  question.  If  test 
Items  are  found  that  are  not  Independent,  the  Items  should  be  revised. 


7-26.  PUT  ITEMS  IN  OPERATIONAL  SEQUENCE.  The  final  activity  of  the 
process  Is  to  place  the  Items  In  an  SU  In  operational  sequence.  In 
developing  this  sequence,  list  the  Items  In  the  order  that  they  would 
occur  on  the  Job. 


! 


I 


I 


\ 


) 
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Example  1 - Detaralnc  Parachute  Ralcaaa-Valght  Setting  for  Equlp- 
■ent  Drop 

Tou  have  Juat  coMpleted  rigging  at  19*500  lb*  load  for  drop  by  six 
Gll-A  parachutes*  The  parachutes  are  rigged  with  5*000  lb*  release 
assesblles.  What  la  the  release  weight? 


1. 

18*000 

lb 

5. 

3*000 

lb 

2. 

14*500 

lb 

6. 

2*500 

lb 

3. 

3*900 

lb 

7. 

2*000 

lb 

4. 

3*500 

lb 

8. 

1*500 

lb 

EXAMPLE:  Pcrforwancc-Baaed  Scorable  Unit 

TASK:  Set  Up  Field  Telephone  and  Contact  Another  Station 
ESSENTIAL  BEHAVTORS: 

Strip  1”  insulation  fros  wire 
Connect  field  wire  to  binding  post 
Install  batteries  (one  4-  up*  other  -f  down) 

*Iurn  circuit  selector  switch  to  LB 
Thrn  buzzer  voluae  control  to  LOUD 
Turn  EXT-INT  switch  to  INT 

*Slgnal  station  by  turning  crank  with  handset  In  retaining  cradle 


\ 


*Key  Behaviors 


Figure  7-5.  Examples  of  Wrinen-Performsnoe  Items  (page  1 of  6). 
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Your  weapon  system  Is  In  a fixed  defense  of  an  airfield.  Communication 
is  by  field  telephone.  Field  wire  has  been  laid  from  Switchboard 
SB-993/GT  to  your  position.  The  Telephone  Set,  TA-312/PT  has  just  been 
Issued  from  supply.  The  power  source  will  be  two  BA-30  batteries. 


1.  How  would  you  prepare  the  field  wire? 


a.  Strip  last  3 Inches 
*b.  Strip  last  inch, 
c.  Leave  insulation  intact. 


2.  How  would  you  Install  the  two  batteries  in  the  TA-312/PT? 


*a.  One  with  positive  (electrode)  end  up,  the  other  with  negative 
(smooth)  end  up. 

b.  Both  with  positive  (electrode)  end  up. 

c.  Both  with  negative  (emooth)  end  up. 


3.  Where  on  the  TA-312PT  would  you  secure  field  wire? 


*a.  Binding  posts, 
b.  External  battery  terminals. 


A.  Which  is  the  correct  position  for  thr  circuit  selector  switch? 


a.  CBS 

b . CB 
*c.  LB 


5.  Which  is  the  correct  position  for  buzzer  volume  control  knob? 


a.  LOW 

b.  Halfway  between  LOW  and  LOUD 
*c . LOUD 


6.  What  is  the  correct  position  for  the  EXT-INT  switch? 


a.  EXT 
*b.  INT 


7.  How  would  you  signal  another  station? 


a.  Press  push-to-talk  switch. 

*b.  Turn  generator  crank  with  handset  securely  In  retaining  cradle, 
c.  T\irn  generator  crank  wtlh  handset  removed  from  retaining  cradle. 


* Key  behaviors 


Figure  7-5.  (page  2 of  6) 
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Example  2 - Determine  Distance  Along  a Route  on  a Map 


You  are  at  point  A (See  map  below.  Figure  7-5)  and  have  been  ordered 
to  move  your  squad  to  a position  at  B.  Since  the  road  Is  secure,  y 'u 
plan  to  move  along  that  route.  What  la  the  distance  In  meters  along 
the  road  from  A to  B. 


Miles 


fidTCRS 


SPi» 


BtAMIl  'W  with  • Mia  •!  VrlttM  Farfei 


FMltim  on  ch«  greunl  to  anotlMr  paiM 
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General  Situation;  You  are  a fire  tean  leader*  Your  squad  leader  has 
Just  told  you  to  take  your  fire  team  to  the  company  trains  (supply  point) 
to  pick  up  rations  and  ammunition  for  the  platoon*  He  hands  you  a map 
and  tells  you  that  you  are  at  Point  A*  He  wants  you  to  go  to  grid 
coordinate  EG  158858*  (A  protractor/grid  coordinate  scale  is  provided 
in  each  test  packet*) 

1*  Special  Situation:  Using  the  map  above,  what  is  the  six-digit 
coordinate  of  Point  "A"? 


a* 

EG 

155856 

*b* 

EG 

155854 

c* 

EG 

154857 

d* 

EG 

153856 

e* 

EG 

153854 

2*  Situation  continued:  What  terrain  feature  is  located  at  grid 
coordinate  EG  158858  (where  you  are  going)? 

a*  Hilltop 
b*  Ridge 
*c*  Valley 
d*  Saddle 
e*  Depression 

3*  Situation  continued:  What  is  the  ground  distance  from  Point  "A" 
to  grid  coordinate  EG  158858? 

a*  350  meters 
b*  450  meters 
*c*  500  meters 

d*  650  meters 

4*  Situation  continued:  In  this  case  the  grid-magnetic  (GN)  angle  is 
so  small  that  the  grid  azimuth  and  the  magnetic  azimuth  is  the  same* 

What  magnetic  azimuth  should  be  followed  going  to  grid  coordinate 
EG  158858? 

a*  28®  d*  43® 

*b*  33®  e*  48® 

c*  38®  f*  53® 

5*  Situation  continued:  You  are  at  the  company  supply  point  packing  up 
your  platoon's  suplies  and  the  first  sergeant  tells  you  your  platoon  has 
Just  moved  to  grid  coordinate  171843  (Point  B).  What  is  the  grid  azimuth 
to  your  platoon's  new  location? 

*a*  125 

b*  140 
c*  305 
d*  320 
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Figure  7-5.  (page  5 of  6) 
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6>  Situation  continued:  What  la  the  elevation  of  the  new  platoon 
position?  (The  contour  Interval  Is  20  feet.) 

a.  720 

b.  880 

c.  900 

d.  920 


7.  Situation  continued:  After  your  return  from  the  company  supply 
point,  your  squad  leader  gives  you  a mission  to  reconnolter  a bridge 
site  located  at  map  coordinate  195855.  The  grid  (map)  azimuth  to  the 
bridge  from  your  location  Is  47.  Since  you  will  be  using  a lensatlc 
compass  you  must  convert  the  grid  (map)  azimuth  to  a magnetic  (compass) 
azimuth.  Using  the  grid-magnetic  angle  shown  below,  what  Is  the  magnetic 
azimuth  you  will  be  using? 


a.  40° 

*b.  43° 

c.  51° 

d.  54° 

8.  If  you  travel  100  meters  In  130  paces,  how  many  paces  would  you  take 
to  travel  800  meters? 

a.  800 

b.  840 

c.  940 

*d.  1040 

9.  Where  should  the  black  Index  line  on  the  lensatlc  compass  be  set  when 
you  navigate  cross  country? 

a.  Directly  over  the  North  arrow 

b.  Directly  over  the  grid  azimuth 
*c.  Directly  over  the  magnetic  azimuth 

d.  Directly  over  the  grid  azimuth  divided  by  3 

e.  Directly  over  the  magnetic  azimuth  divided  by  3 


Figure  7-5.  (page  6 of  6) 
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CHAPTER  8 

VALIDATE  WRITTEN  COMPONENT 


Section  I.  THY  OUT  WRITTEN  SU 


8-1.  INTRODUCTION.  The  tryout  Is  a systematic  way  to  have  experts  In 
the  MOS  review  the  WC  to  be  sure  each  SU  Is  consistent  with  doctrine. 
You  must  test  experts  and  collect  their  opinions  of  the  SU  before  sub- 
mitting the  WC  to  ITED. 


8-2.  ADMINISTER  SU  TO  TEST  DEVELOPERS.  Before  you  begin  testing  experts,  | 

two  or  three  people  who  helped  develop  an  SQT  should  take  each  SU.  Test  ] 

developers  do  not  have  to  be  qualified  In  the  MOS  covered  by  the  SQT. 

1 

I 

8-3.  IS  THE  SU  ACCEPTABLE?  Ask  each  test  developer  to  suggest  changes 
In  the  SU.  Test  developers  who  take  the  test  should  look  for  cues  within  j 

each  Item  or  In  other  Items,  check  readability  of  Items  and  check  clarity  I 

of  Illustrations.  ^ 


8-4.  REVISE  THE  SU.  Weigh  suggestions  of  other  test  developers.  If 
a suggestion  makes,  sense,  change  the  Item  or  Items.  Have  test  develop- 
ers review  the  revised  SU. 


8-5.  SELECT  AT  LEAST  THREE  EXPERTS. 

a.  In  this  case  "experts"  are  people  who  you  are  sure  can  do  the  task  1 

covered  by  an  SU.  They  do  not  have  to  be  In  the  skill  level  for  the  SQT.  ' 

In  fact,  experts  usually  will  be  senior  NCO's  or  Instructors  In  the  MOS. 

One  person  may  be  an  expert  for  several  tasks  covered  by  an  SQT.  Test 
developers  should  not  act  as  experts  for  this  part  of  the  tryout. 

i 

b.  The  quality  of  the  tryout  depends  on  the  level  of  expertise  of 

the  soldiers  who  take  the  SU.  Do  whatever  you  can  to  get  high-level  I 

performers  for  the  tryout.  If  you  have  doubt  about  qualifications  of  a ' { 

potential  expert.  Interview  his  peers  and  supervisor.  Ask  the  peers  and  > 

supervisor  how  well  the  expected  expert  can  perform  the  specific  task*  J 

If  they  say  the  expected  expert  can  do  the  task  very  well,  consider  In-  | 

eluding  him  In  the  tryout.  If  they  say  the  expected  expert  can  do  the 
task  only  fairly  well  or  worse,  do  not  Include  him.  j 

! j 

i 

( 
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c*  After  you  locate  aeverel  expected  experta,  give  thete  a copy  of  the 
teak  anelyais  end  have  them  rate  their  ability  to  perform  the  task. 
When  collecting  aelf-ratlngs,  emphasize  Importance  of  accuracy.  Be 
sure  expected  experta  know  there  Is  no  benefit  to  them  or  the  SQT  If 
they  overrate  their  ability.  Use  one  of  these  questions  for  self- 
ratings.  The  question  to  choose  depends  on  whether  the  task  Is 
performed  normally  under  supervision. 

(1)  How  well  can  you  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Not  very  %rell. 

2.  Fairly  well. 

3.  Very  well. 


(2)  To  perform  the  complete  task,  how  much  technical 
guidance  would  you  require  from  someone  trtio  knows 
how  to  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Lots  of  guidance. 

2.  Some  guidance. 

3.  No  guidance. 

For  this  tryout,  experts  are  people  who  choose  No.  3,  "Very  well"  or  "No 
guidance." 


8-6.  ADHINISTER  SU  TO  EXPERTS.  Test  experts  one  at  a time.  Make  condi- 
tions as  similar  as  possible  to  field  conditions.  Use  the  sasM  Instruc- 
tions and  Illustrations  as  when  testing  for  record.  Allow  time  for  the 
expert  to  answer  Items  and  review  answers. 


8-7.  EXPERTS  PASS  EACH  ITEM?  One  minimum  requirement  of  an  SQT  written 
SU  Is  that  experts  on  a task  consistently  pass  Items  based  on  that  task. 
If  an  expert  misses  any  Item,  find  out  why.  The  expert  may  have  for- 
gotten how  to  do  the  task  or  'dte  task  may  have  chenged.  Otherwise, 
revise  the  Item  to  eliminate  tItM  characteristic  that  led  to  the  Incorrect 
answer . 


8-8.  IS  THE  SU  ACCEPTABLE  TO  EXPERTS?  Have  the  experts  complete  the 
questionnaire  In  Figure  8-2  for  each  SU  Immediately  after  they  finish  the 
SU.  The  questions  ask  experts  to  evaluate  criticality  of  the  task, 
fairness  of  the  Items,  and  Inclusiveness  of  the  SU.  If  more  than  one- 
third  of  the  experts  think  the  task  is  non-crltlcal,  an  Item  Is  unfair 
or  the  SU  Is  Incomplete,  revise  the  SU.  Questions  relevant  to  this 
decision  arc  marked  with  an  asterisk  (*). 


f 
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8-  9*  REVISE  THE  SU.  Your  primary  concern  during  this  tryout  Is  that 
the  SU  Is  consistent  with  doctrine.  If  experts  select  an  Incorrect 
alternative,  check  the  alternative  to  be  sure  It  Is  a real-world  alternative 
and  Incorrect.  If  experts  thlr.k  the  task  Is  non-crltlcal,  consider 
dropping  It  from  the  component.  If  experts  think  an  Important  behavior 
has  been  left  out  of  the  SU,  develop  an  Item  for  that  behavior.  Have 
the  experts  review  the  revised  SU. 


8-10.  ADD  THE  SU  TO  THE  WC.  A written  SU  that  experts  consistently  pass 
and  consider  acceptable  meets  first  requirements  for  Inclusion  In  the 
SQT.  Add  it  to  the  component  and  repeat  the  procedure  for  each  remaining 
SU. 


8-11.  SUBMIT  WC  TO  ITED.  Submit  the  following  things  to  ITED: 

• All  SU  In  the  WC. 

•A  summary  of  the  tryout. 

•The  task  analysis  data  for  each  task  covered. 

•Administrative  Instructions. 

• The  SQT  notice  for  the  WC. 

•Choice  of  two  SU  to  be  validated  against  hands-on  (Figure  8-4). 
Submit  the  WC  at  the  same  time  you  submit  the  HOC. 


|0nce  the  WC  has  been  approved  by 
I ITED  the  content  of  each  item 
Iwlthln  each  SU  Is  fixed  and  can- 
|not  be  changed  without  approval 
I from  ITED.  Items  found  to  be 
lunsatlsfactory  during  validation 
(will  be  revised  to  Improve  them, 
(but  not  to  extent  of  changing 
Icontent.  Unsatisfactory  Items  can- 
|not  be  deleted  without  approval. 
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INSTRUCTIONS:  Please  take  attached  scorable  unit  and  answer  questions  to 
the  best  of  your  ability.  Now  look  at  this  task  In  the  applicable 
Soldier's  Manual  and  answer  the  following  questions: 


PART  I - Task  Analysis 

*1.  la  this  task  a critical  task  for  the  MOS  and  skill  level  for  which 
it  is  written?  If  no,  explain. 


a.  Yes 


b.  No 


c.  Not  sure 


*2.  Have  any  steps  (performance  measures)  been  left  out  of  the  Soldier's 
Manual?  If  yes,  explain. 


a.  Yes 


b.  No 


c.  Not  sure 


*3.  Have  any  unnecessary  steps  (performance  measures)  been  Included? 
If  yes,  explain. 


a.  Yes 


b.  No 


c.  Not  sure 


Figure  8-2.  Subiect-Matter-Expert  Questionnaire  (To  be  used  in  review  of  written  scorable 
unit)  (page  1 of  5) 


I 1 
! 

i 
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PART  II:  Writccfi  Scorablc  Unit  Evaluation 

r 

INSTRUCTIONS:  Now,  ravlaw  tha  acorabla  unit  and  answar  tha  following 
quaatlons: 

I 

4.  How  nany  quaatlona  (parfoimanca  aaaauraa)  ahould  a aoldlar  anawar 
corractly  In  ordar  to  ba  glvan  a GO  on  tha  taak: 

of 


5.  Do  you  think  you  racalvad  a GO  (paaa)  or  NO-GO.  (fall)  on  tha  unit? 

j a«  Go  b*  NO>^K>  c«  Not  aura 

1 

rt 

*6.  la  thla  unit  raallatlc?  Clrcla  ona*  If  no,  atata  why. 

a.  Yaa  b.  No  c.  Not  aura 


*7.  la  the  unit  doctrlnally  corract?  If  no,  atata  why. 

a.  Tea  b.  No  c.  Not  aura 


8.  Would  an  llluatratlon  or  additional  llluatratlona  aaka  tha  unit 
(or  any  apaclflc  Iten)  eaaler  to  undaratand?  (If  yaa,  what  Uluatra- 
tlon(a)  ahould  be  added,  and  where? 

a.  Tea  b.  No  c.  Not  aura 


1 
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*9*  If  the  unit  h«a  an  illuatratlon  or  illustrations,  was  it  (or  they) 
easy  to  understand?  If  no,  tell  why. 


a.  Tea 


b.  No 


c.  Not  sure 


*10.  Here  words  or  phrases  in  the  unit  difficult  to  understand?  If 
yes,  which  ones? 


a.  Yes 


b.  No 


c.  Not  sure 


*11.  Did  you  understand  what  you  were  aaked  to  do  in  each  Itea  of  the 
unit?  If  no,  tell  why. 


a.  Yes 


b.  No 


c.  Not  sure 


*12.  Do  you  agree  with  the  correct  solution(s)?  If  no,  tell  why. 
a.  Yes  b.  No  c.  Not  sure 


13.  In  the  following  space,  tell  which  errors  a soldier  aost  often  aakes 
in  perforalng  the  task  on  which  this  scorable  unit  is  based. 


*14.  Review  the  answers  to  each  question.  Are  all  of  the  answers 
possible  "real-world"  alternatives?  If  no,  which  answers  are 
unrealistic? 


a.  Yes 


b.  No 


c.  Not  sure 


i 


I 

I 


I 
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INSTRUCTIONS:  These  questions  apply  to  your  overall  evaluation  of  the 
scorable  unit. 

*15.  Should  questions  (Items)  be  added  to  the  scorable  unit?  If  yes, 
explain. 

a.  Yes  b.  No  c.  Not  sure 

*16.  Should  questions  (Items)  be  deleted  from  the  scorable  unit?  If  yes, 
explain. 

a.  Yes  b.  No  c.  Not  sure 


*17.  As  the  scorable  unit  la  now  written,  do  you  think  It  Is  fair  measure 
of  a soldier's  ability  to  perform  the  task  on  which  It  is  based?  If  no, 
explain. 

a.  Yes  b.  No  c.  Not  sure 


18.  If  your  recommendations  are  Included  In  a revision  of  the  scorable 
unit,  will  It  then  be  a fair  measure  of  a soldier's  ability  to  perform 
the  task  on  which  It  Is  based?  If  "no",  explain. 

a.  Yes  b.  No  c.  Not  sure 


Figure  8-3.  (page  4 of  5) 
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INSTRUCTIONS:  Now  read  the  portion  of  the  Soldier's  Manual  in  which  this 
scorable  unit  Is  based  and  answer  the  following  questions. 

19.  Can  a soldier  answer  questions  In  the  scorable  unit  Just  by  studying 
the  applicable  Soldier's  Manual? 

a.  Yes  b.  No  c.  Not  sure 


20.  (If  your  answer  Is  yes,  to  question  19,  skip  this  question).  If 
you  answered  no  or  not  sure,  look  at  the  reference  materials.  Can 
the  soldier  answer  questions  In  the  scorable  unit  correctly  by  studying 
or  using  referenced  materials  (FMs,  TMs,  AR's  etc.)?  If  no,  tell  why. 

a.  Yes  b.  No  c.  Not  sure 

1 

I 21.  Are  task,  conditions,  and  standards  tested  In  the  scorable  unit  the 

same  as  task  conditions  and  standards  In  the  applicable  Soldier's 
j Manual?  If  no,  tell  why. 

a.  Yes  b.  No  c.  Not  sure 


INSTRUCTIONS:  If  applicable,  review  the  TEC  LessonCs)  which  cover 
the  task  on  which  this  scorable  unit  Is  based  and  answer  the  following 
questions. 

22.  Are  the  task  conditions  and  standards  In  this  scorable  unit  the  same 
as  the  task,  conditions  and  standards  In  the  applicable  TEC  lesson?  If 
no,  tell  why. 

a.  Yes  b.  No  c.  Mot  sure 
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Section  11.  TRY  OUT  WRITTEN  COMPONENT 


8-*. 2.  INTRODUCTION.  Dlls  tryout  will  be  first  chance  to  find  whether 
the  whole  component  la  appropriate  to  a aaaple  of  reprcacntatlve  aoldlera 
In  the  MOS. 


8-13.  SELECT  FIVE  SOLDIERS.  After  ITED  approvea  the  WC,  conduct  an 
Inforaal  tryout  of  the  conponent.  The  flrat  atep  la  to  locate  five 
aoldlera  at  the  aklll  level  of  the  MOS  covered  by  the  SQT. 


8-14.  ADMINISTER  WC  TO  SOLDIERS.  Teat  aoldlera  on  the  SU  that  are 
critical  to  their  aklll  level.  Make  condltlona  aa  alallar  aa  poaslble 
to  the  field  conditions.  Use  the  same  Illustrations  and  Instructions 
as  when  tasting  for  record.  Allow  soldiers  time  to  answer  each  question 
and  to  review  their  answers. 


8-15.  INTERVIEW  SOLDIERS.  After  soldiers  finish  the  component,  ask  each 
one  If  Instructions  or  Items  ware  hard  to  understand. 


1 8-16.  IS  WC  ACCEPTABLE?  This  Informal  tryout  should  answer  two  quas-  ! 

I tlons:  ' 

I ‘ 

I • Can  soldiers  at  the  skill  level  understand  the 

I Instructions,  questions  and  altarnatlvea  on  the 

I test? 

1 

I • Can  soldiers  at  the  aklll  level  finish  the  test 

In  the  time  allowed?  Since  this  tryout  does  not 
Include  Items  for  the  next  highest  aklll  level, 
anything  over  three  hours  should  alert  you  to  a 
possible  problem. 

! 

8-17.  REVISE.  Make  only  minor  revisions  based  on  this  tryout.  Add 
Illustrations -or  reword  confusing  Items;  do  not  change  content  of 
any  SU.  Have  soldiers  In  sample  check  revisions  to  assure  you  have 
eliminated  the  source  of  confusion. 


1 
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ITED 

Approves  WC 


Select  more  soldiers 

80»  \ 

^ cxo.  oarf.  pea?  \ No  ^ 

Revise  hands-on  SU 

(par.  8-25) 

(par.  8-23) 

/ 4perf.  \ 
■rtd  4 norvperf.? 
V(|«».i.*4)  > 


Index  poiWve?x^^_^ 
tPW.  t-aW 

( •^) 

Figure  84.  Validate  Two  Written  SU  Against  Hands-On 

8-12 


Reviw  item 
(par.  8-29) 


i 

li 


I 


! 
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Section  III.  VALIDATE  TWO  WRITTEN  SU  AGAINST  HANDS-ON 


8-18.  INTRODUCTION.  Each  option  In  the  validation  plan  requires  you  to 
select  a group  of  soldiers  who  can  do  a task  (performers)  and  a group 
who  cannot  do  a task  (non-performers).  You  will  then  compare  the  two 
groups  on  their  written  test  scores  for  that  task. 

Options  vary  in  the  way  you  identify  performers  and  non-performers. 
The  best  option  for  identifying  performers  and  non-performers  is  to  give 
a group  of  soldiers  a hands-on  test  for  the  task.  Unfortunately  that  is 
also  the  most  expensive  option. 

It  is  not  likely  that  any  TDA  will  have  the  resources  to  validate  all 
written  SU  against  hands-on  tests.  But  each  TDA  must  validate  at  least 
two  written  SU  against  hands-on  tests.  TDA  will  thus  build  up  a reser- 
voir of  written  SU  validated  against  the  strongest  possible  criterion. 


8-19.  DEVELOP  TWO  HANDS-ON  SU.  Choose  two  tasks  that  you  think  will  be 
in  the  SQT  in  the  future.  Develop  a hands-on  test  for  each  •‘ask.  Follow 
the  procedure  in  Chapter  5. 


8-20.  SELECT  TEN  SOLDIERS.  Select  10  soldiers  in  the  MOS  the  SQT 
addresses.  Assure  a range  of  ability  among  the  soldiers  by  getting  five 
soldiers  you  expect  to  be  performers  (high  rank  or  AIT  instructors)  and 
five  you  expect  to  be  non-performers  (low  rank  or  AIT  students). 

If  you  cannot  get  10  soldiers,  coordinate  with  ITED  for  permission 
to  reduce  the  size  of  the  sample. 


8-21.  ADMINISTER  HANDS-ON  TEST.  Test  each  soldier  you  select  on  the  | 
hands-on  SU.  Soldiers  who  receive  a GO  are  performers;  soldiers  who  I | 
receive  a NO-GO  are  non-performers.  I 


8-22.  DO  80Z  OF  EXPECTED  PERFORMERS  PASS?  If  the  people  you  expect  to 
be  able  to  do  the  task  fall  the  hands-on  test,  something  may  be  wrong 
with  the  hands-on  test.  Discuss  the  test  with  expected  performers  to 
find  out  why  they  failed.  If  expected  performers  insist  that  the  test 
is  wrong,  revise  the  test. 


8-13 


_ 
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8-23.  REVISE  HANDS-ON  SU. 

a.  If  •xpectad  parforaara  do  not  paaa  tha  SU  and  insist  that  thay 
raally  ara  parfornars,  two  things  night  ba  wrong: 

• Iha  parfomanca  maasuras  nay  ba  too  inpracisa. 

• Tha  tast  nay  ba  basad  on  incorract  doctrina. 

On  tha  othar  hand,  tha  tast  may  ba  pointing  up  an  arror  so  conmon  that 
avan  axpactad  parfornars  commit  it. 

b.  If  you  find  that  tha  doctrina  for  tha  tast  is  wrong,  chack  tha 
writtan  SU  for  that  task.  If  it  is  also  basad  on  incorract  doctrina, 
ravioa  tha  writtan  SU. 


8-24.  DOES  SAMPLE  INCLUDE  AT  LEAST  FOUR  PERFORMERS  AMD  FOUR  NON- 
PERFORMERS? Tha  analysis  for  this  validation  raquiras  that  aach  group 
(parfomar  and  non-parfomar)  ba  raprasantad  in  tha  sampla.  At  laast 
four  of  aach  and  a total  of  tan  will  naat  tha  raquiramants.  If  more 
ara  naadad  in  aithar  group,  aalact  mora  soldiars. 


8-25.  SELECT  MORE  SOLDIERS.  If  aithar  group  is  undar-raprassntad , 
salact  mora  soldiars  in  tha  MOS  and  administar  tha  hands-on  SU  to  them. 


8-26.  ADMINISTER  WRITTEN  SU.  Hava  tha  10  soldiars  in  tha  validation 
sampla  answer  tha  questions  in  tha  writtan  SU.  Make  conditions  as  close 
as  possible  to  conditions  for  operational  administration.  Use  tha  sane 
Illustrations  and  instructions.  Allow  soldiars  as  much  time  as  needed  to 
answer  aach  question  and  to  review  their  answers. 


8-27.  CONFUTE  AGREEMENT  INDEX. 

a.  First,  score  tha  writtan  SU.  Apply  the  scoring  rules  you  devel- 
oped for  the  SU. 

b.  Second,  prepare  a 2x2  table  for  each  item.  Tha  table  should 
contain  tha  following  information t 
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HANDS-ON  TEST 


WRITTEN  ITEM 

Paaa 

Fall 

|a  * Nuaber  of  aoldlera  who 

1 b - 

nuaber  of  aoldlera  | 

Paaa 

I paaa  handa-on  teat  and 

1 who 

fall  handa-on  teat  H 

Ipaaa  written  Itea. 

1 and 

paaa  written  Itea.  | 

|c  ■ Nuaber  of  aoldlera  who 

1 d - 

nuaber  of  aoldlera  | 

Fall 

Ipaaa  handa-on  teat  and 

1 who 

fall  handa-on  teat  | 

jfall  %rrltten  Item. 

1 and 

fall  written  Itea.  j 

Agreeaent  Index  - (a  x d)  - (b  x c) 


c.  Iblrd,  coapute  the  Agreeaent  Index  [AI  - (a  x d)  - (b  x c)]  for 

each  table.  ‘ 

d.  For  exaaple,  aaauae  that  four  of  alx  perfomcra  (GO  on  handa-on 
teat)  pasa  an  Itea  that  only  one  of  four  non-perforaera  (NO-GO  on  handa- 
on  teat)  paaaea.  The  2x2  table  would  look  like  thla: 


I 


1 i; 

1 

l_ 

1 

_l_ 

^ 1 

2 

1 

_l_ 

3 1 

; The  equation  for  the  table  la: 

AI  - 

(4  X 

3) 

-(2x1) 

AI  - 

12  - 

2 

AI  - 

10 

The  agreeaent  Index  of  10  Indlcatea  an  acceptable  level  of  agreeaent  be- 
tween perforaance  on  the  handa-on  teat  and  perforaance  on  the  Itea.  Any 
Index  greater  than  zero  la  acceptable. 


8-28.  IS  INDEX  POSITIVE?  If  pcrforaers  do  better  on  an  Itea  than  non- 
perforaera  do,  the  Index  will  be  higher  than  zero.  If  the  Index  la  zero 
or  a negative  nuaber,  there  la  probably  aoaethlng  wrong  with  the  Itea. 
Such  Iteaa  auat  be  revlaed.  If  the  Index  la  poaltlve,  the  Itea  la  valid. 
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8-29.  REVISE  ITEM. 

a.  If  the  people  who  developed  the  ?U  have  followed  development  proce- 
dures, minor  revisions  will  probably  make  Che  items  acceptable.  If  the 
item  is  too  hard  (more  perforn\ers  fall  than  pass  it),  check  the  wording 

of  the  item,  check  the  accuracy  of  the  correct  alternative,  and  consider 
adding  Illustrations  to  increase  clarity. 

If  the  Item  Is  too  easy  (non-performers  pass  It  at  least  as  often  as 
performers),  check  for  unwanted  grammatical  cues,  check  for  cues  In 
other  items  that  may  give  away  the  Item  in  question,  and  check  Che 
possibility  that  a likely  wrong  alternative  has  been  left  out  of  the  Item. 

b.  If  the  Item  does  not  have  a superficial  deficiency,  you  may  need 
to  make  a major  revision.  First  analyze  responses  of  the  performers  to 
find  which  %nrong  alternative  they  usually  select.  Be  sure  that  alter- 
native Is  a real-world  alternative  and  is  wrong.  Then  check  the  task 
analysis  data  to  be  sure  that  the  behavior  the  Item  measures  Is  a key 
behavior. 

If  Che  behavior  is  not  key,  you  may  be  able  to  coordinate  with  ITED 
to  delete  it  from  the  SU.  If  after  careful  study  you  can  find  no  fault 
with  the  Item  and  the  behavior  Is  key  to  successful  performance  of  the 
task,  accept  the  Item  as  Is. 

c.  Repeat  the  validation  process  for  each  Item  you  revise. 


Section  IV.  SELECT  VALIDATION  OPTION  FOR  WC 

8-30.  INTRODUCTION.  Six  options  are  available  for  validating  the  WC. 

The  option  you  choose  will  depend  on  availability  of  Incumbents  in  the 
NOS,  supervisors  of  Incumbents  In  the  MOS,  or  experts  In  the  MOS.  The 
procedure  for  selecting  a validation  option  is  based  on  two  assumptions. 

The  first  assumption  Is  that  the  self-rating  option  with  30  soldiers 
Is  the  most  practical  option.  We  know  more  about  It  than  the  other  two 
and  It  poses  the  fewest  logistical  problems.  The  second  assumption  Is 
that  most  TDA  will  prefer  to  conduct  the  validation  at  their  Installation. 

Therefore  the  procedure  recommends  considering  sll  three  local  options 
before  considering  the  TDY  options.  If  you  have  TDY  funds  available,  you 
may  wish  to  alter  priority  of  the  options  and,  for  example,  select  the 
TDY  self-rating  option  before  trying  the  local  supervisor  or  local  panel- 
of-experts  options. 

The  Intent  of  this  procedure  Is  to  give  TDA  flexibility  In  validating 
the  WC.  You  may  combine  the  methods  to  validate  the  complete  component. 

If  even  this  flexibility  is  not  enough,  coordinate  with  ITED. 
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8-31.  ARE  AT  LEAST  30  SOLDIERS  AVAILABLE  LOCALLY?  Soldiers  for  the 
validation  should  be  Incumbents  In  the  >K)S  or  a mixture  of  AIT  students 
and  Incumbents.  The  goal  la  a spread  of  ability  to  perform  the  tasks. 
If  a large  group  of  soldiers  with  a spread  of  ability  Is  available  at 
your  Installation,  select  the  local  self-rating  option. 


8-32.  ARE  5 TO  10  SOLDIERS  AVAILABLE  LOCALLY?  If  a small  group  of 
soldiers  with  a spread  of  ability  Is  available  at  your  Installation,  you 
may  be  able  to  select  either  the  local-supervisor  option  or  the  local 
panel-of-experts  option.  If  even  a small  group  of  soldiers  with  a range  ' 
of  ability  is  not  available  at  your  Installation,  consider  the  TDY 
options. 

8-33.  ARE  SUPERVISORS  AVAILABLE  LOCALLY?  If  the  small  group  of  soldiers 
at  your  Installation  has  at  least  one  Immediate  supervisor  yot  can  inter- 
view, select  the  local  supervisor  option. 

If  no  supervisor  Is  available  for  the  small  group  of  soldiers, 
consider  the  local  expert  option. 


8-34.  ARE  THREE  EXPERTS  AVAILABLE  LOCALLY?  For  this  decision,  an  expert 
Is  a person  generally  acknowledged  to  be  a high-level  performer  on  a task, 
for  example,  an  AIT  instructor.  Do  not  consider  other  test  developers 
experts  for  this  decision.  If  at  least  three  experts  on  each  task  in  the 
WC  are  available  at  your  Installation,  select  the  local  panel  of  experts 
option.  If  not  enough  experts  can  form  a panel  for  each  SU,  consider  TDY 
options. 


8-35.  ARE  30  SOLDIERS  NEAR?  In  this  case  "near"  means  an  Installation 
close  enough  to  visit  that  will  allow  access  to  soldiers.  If  you  visit 
another  Installation  to  conduct  the  validation  and  a large  group  of 
soldiers  with  a range  of  ability  Is  available,  select  the  TDY  self-rating 
option.  The  procedure  for  this  option  Is  the  same  as  for  the  local  self- 
rating option.  If  a large  group  of  soldiers  with  a range  of  ability  Is 
not  available,  consider  other  TDY  options. 


8-36.  ARE  5 TO  10  SOLDIERS  NEAR?  If  a small  group  of  soldiers  with  a 
spread  of  ’ability  is  at  an  Installation  you  visit,  you  may  select  the 
TDY  supervisor  or  TDY  panel  of  experts  option.  If  even  a small  group 
of  soldiers  is  not  available  at  a nearby  Installation,  contact  ITED. 
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8-37.  ARE  SUPERVISORS  NEAR?  If  the  small  group  of  soldiers  at  the  near- 
by Installation  has  at  least  one  Immediate  supervisor  you  can  Interview, 
select  the  TOY  supervisor  option.  The  procedure  for  this  option  Is  the 
same  as  for  the  local  supervisor  option  described  earlier.  If  there  Is 
no  available  supervisor  for  the  small  group  of  soldiers,  consider  the 
TDY  expert  option. 


8-38.  ARE  THREE  EXPERTS  NEAR?  If  at  least  three  high  level  performers 
on  each  task  In  the  UC  are  available  at  a nearby  Installation,  select  the 
TDY  expert  option.  The  procedure  for  this  option  Is  the  same  as  for  the 
local  expert  option  described  earlier.  If  sufficient  experts  are  not 
available,  contact  ITED. 


Section  V.  VALIDATE  WRITTEN  SU  - SELF  RATINGS 


8-39.  SELECT  30  SOLDIERS. 

a.  Soldiers  for  this  option  nay  be  Incumbents  In  the  MOS,  or  AIT  stu- 
dents or  a combination  of  Incumbents  and  AIT  students.  Incumbents  In  the 
MOS  do  not  havo  to  be  at  the  skill  level  for  the  SQT,  In  fact  a range  of 
skill  levels  Is  desirable  since  '’hat  helps  assure  a spread  of  ability  to 
perform  the  tasks. 

b.  Kax’e  each  candidate  for  the  sample  rate  his  or  her  own  ability  to 
perfoi"  t-he  tasks  covered  by  the  WC.  When  you  collect  the  self-ratings, 
emphasize  the  Importance  of  accuracy.  Be  sure  expected  experts  know 
there  Is  no  benefit  to  then  or  the  SQT  If  they  overrate  their  ability. 
C'-'ve  each  soldier  a detailed  description  of  the  task  that  specifies 

.he  task  conditions  and  standard.  Then,  ask  each  soldier  one  of  the 
following  questions.  The  question  to  ask  depends  on  whether  the  task  Is 
normally  performed  under  supervision. 

(1)  How  well  can  you  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Not  very  %rell 

2.  Fairly  well. 

3«  Very  well. 

(2)  To  perform  the  complete  task,  how  much  technical 

guidance  would  you  require  from  someone  who  knows 

how  to  pei^fom  the  task? 

0.  Not  familiar  with  the  task. 

1.  Lots, of  guidance. 

2.  Some 'guidance. 

3.  No  guidance. 
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c>  The  self-ratings  Identify  groups  of  performers  and  non-performers 
for  each  SU>  Soldlera  who  rate  themselves  low  (0  or  1)  are  In  the  non- 
performer group:  soldiers  who  rate  themselves  high  (2  or  3)  are  In  the 
performer  group*  A soldier  may  be  In  the  performer  group  for  one  task 
and  In  the  non-performer  group  for  another  task. 


8-40.  DOES  SAMPLE  HAVE  AT  LEAST  10  PERFORMERS  AND  10  NON -PERFORMERS? 

One  requirement  for  a valid  Item  Is  that  a greater  proportion  of  perform- 
ers than  non-performers  pass  the  Item.  To  tell  If  that  happens,  you 
must  be  sure  that  the  sample  of  soldiers  Includes  performers  and  non- 
performers. Also  assure  that  the  size  of  each  group  la  large  enough 
to  reduce  Impact  of  Incorrect  self-ratings.  At  least  10  per  group  and 
a total  of  30  will  enable  you  to  be  confident  In  the  computed  Agreement 
Index . 


8-41.  SELECT  MORE  SOLDIERS.  If  you  have  less  than  10  performers  or  non- 
performers, add  soldlera  and  collect  self-ratings  until  you  have  at  least 
10  soldiers  per  group  and  a total  of  30. 


8-42.  ADMINISTER  THE  SCORABLE  UNIT.  Have  soldiers  In  the  sample  answer 
the  questions  In  the  SU.  Make  condltlona  similar  as  possible  to  field 
conditions.  Use  Illustrations  and  Instructions  that  will  be  used  when 
you  test  for  record.  Allow  soldiers  as  much  time  as  they  need  to  answer 
each  question  and  to  review  their  answers. 


8-43.  DETERMINE  ACCEPTABILITY.  After  soldiers  finish  the  SU,  have  them 
fill  out  Question  Sheet  2 (Figure  8-7).  The  questions  concern  complete- 
ness and  fairness  of  the  SU.  Although  acceptability  to  non-performers  Is 
not  a criterion  for  a valid  Item,  performers  and  non-performers  should 
complete  the  (Question  Sheet  for  two  reasons: 

• Feedback  from  non-performers  will  be  helpful 
If  you  need  to  revise  an  Item. 

• Soldiers  In  the  sample  will  probably  cooperate 
better  If  you  treat  all  soldiers  alike  regard- 
less of  their  self-ratings. 


8-44.  DO  80Z  OF  PERF(«MERS  ACCEPT  EACH  ITEM?  The  first  requirement  for 
an  SQT  Item  Is  that  aoldlers  who  say  they  can  do  a task  Chink  the  Item  Is 
a fair  way  to  find  whether  a person  can  do  an  Important  part  of  the  task. 
If  more  than  20X  of  the  performers  say  a specific  Item  Is  Irrelevant  or 
unfair,  revise  the  Item. 
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NAME 


EVALUATION  OF  WRITTEN  SCORABLE  UNITS 


Each  Item  should  be  based  on  a behavior  (decision  or  action)  essential  to 
performing  the  task. 

Yes  No 

1.  Does  passing  each  Item  correspond  to  a behavior 

(decision  or  action)  necessary  for  task  performance?  

If  not,  please  specify  which  Items  should  be 
revised : 

2.  Do  the  Items  for  each  Key  Behavior  fully  measure 
that  point?  (Key  Behaviors  have  special  conse- 
quences to  personnel  or  equipment  or  are  frequent 

sources  of  failure)  


If  not,  please  specify  which  items  should  be: 
a.  Added 

Which  one(8)? 
b*  Revised 

Which  one(8),  and  how? 

3*  Based  on  this  Scorable  Unit,  would  you  be  confi- 
dent that  soldiers  who  answer  Items  correctly 
are  able  to  perform  the  task? 
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8-45.  RECORD  SU  PERFORMANCE. 

a.  Clear  your  desk,  sharpen  four  pencils,  check  your  eraser  and  tell 
your  secretary  to  hold  all  calls  from  anyone  who  does  not  wear  a star. 

Now  the  fun  begins. 

b.  First,  score  the  tests.  Apply  the  scoring  rules  you  developed  for 
each  Item  In  the  SU. 

c.  Second,  separate  the  answer  sheets  for  the  performers  (self- 
ratlug  of  2 or  3)  from  the  answer  sheets  for  the  non-perforaers  (self- 
rating of  0 or  1).  Remember,  a soldier  may  be  a performer  on  some  tasks 
and  a non-performer  on  another;  therefore  check  the  self-ratings  for 
each  SU. 

d.  Third,  for  each  SU  list  the  self-rating,  performance  per  Item, 

and  total  Items  passed  for  each  performer.  The  Ordnance  School  has 
developed  an  Item  Analysis  form  you  can  use  to  make  the  list  (Figure 
8-8).  Enter  a "1"  for  each  Item  a soldier  passes  and  a "0"  for  each 
item  he  falls.  (You  could,  of  course,  use  "P"  and  "F"  but  numbers  are 
more  scientific.)  ' 

e.  Fourth,  record  the  total  number  of  performers  who  pass  each  item 
and  the  number  who  fall  each  Item. 

f.  Fifth,  list  the  self-rating,  performance  per  item  and  total  Items 
passed  for  each  non-performer. 

g.  Sixth,  record  the  number  of  non-performers  who  pass  each  item  and 
the  number  who  fall  each  Item. 


8-46.  DO  MORE  PERFORMERS  PASS  THAN  FAIL  EACH  ITEM7  The  second  require- 
ment for  an  SQT  Item  Is  that  more  soldiers  who  say  they  can  perform  the 
task  pass  the  Item  than  fall  It.  If  an  Item  does  not  meet  this  require- 
ment, the  Item  probably  requires  knowledge  or  a skill  not  part  of  the 
Job.  Revise  such  Items  to  eliminate  artificial  difficulty. 


8-47.  OemPUTE  ITEM-RATING  AGREEMENT. 

a.  The  third  requirement  for  an  SQT  Item  Is  that  performers  do  better 
on  the  Item  than  non-performers  do.  You  can  find  whether  that  happens 
by  computing  an  Agreement  Index  betvwen  performance  on  the  Item  (pass/ 
fall)  and  self-ratings  on  the  task  (performer /non-performer). 
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b.  The  first  step  In  computing  sn  item-rating  Agreement  Index  is 
to  prepare  a 2x2  table  for  the  task.  The  table  should  contain  the 
following  information  from  the  summary  of  performance: 


SELF-RATING 


Performer 


I a - Number  of  soldiers  who 
Pass  Irate  themselves  as  perform- 

|ers  and  pass  item 

|c  - Number  of  soldiers  who 
Fall  Irate  themselves  as  perform- 

|ers  and  fail  item 

Agreement  Index  ■ (a  x d)  - (b  x c) 


Non-Performer 

b - Number  of  soldiers  who 
rate  themselves  as  non- 
performers and  pass  item 

d * Number  of  soldiers  who 
rate  themselves  as  non- 
performers and  fall  item 


c.  The  second  step  la. to  apply  the  formula:  Agreement  Index  > (a  x d)  - 
(b  x c). 

d.  For  example,  consider  item  I from  Figure  8-8.  Eleven  performers 
passed  th«  item  and  four  failed  it;  five  non-performers  passed  the  item 
and  ten  failed  it.  The  2x2  table  is: 


The  equation  for  the  table  is:  AI  > (11  x 10)  -(4x5) 

AI  - 110  - 20 
AI  - 90 

The  Agreement  Index  of  90  irdicates  an  acceptable  level  of  agreement 
between  performance  and  self-ratings;  in  fact,  any  index  greater  than 
zero  is  acceptable. 
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8-48.  COMPUTE  ITEM-SU  AGREEMENT. 

a.  Ihe  fourth  requirement  for  an  SQT  Item  la  that  soldiers  who  pass 
the  Item  also  pass  the  other  Items  In  the  SU.  The  way  to  check  whether 
that  happens  Is  to  compute  the  Agreement  Index  between  the  Item  and  the 
SU.  The  procedure  to  compute  the  Agreement  In<*ex  depends  on  whether  the 
SU  contains  up  to  four  Items  or  more  than  four  Items. 

b.  If  the  SU  has  four  or  less  Items,  compute  Agreement  Indexes  for 
all  possible  pairs  of  Items  In  the  SU.  A two-item  SU  would  have  one 
pair;  a three-item  SU  would  have  three  pairs;  and  a four-item  SU  would 
have  six  pairs.  Unless  you  really  like  to  do  things  like  this,  do  not 
compute  agreement  for  Items  already  marked  for  revision.  The  2x2  table 
for  calculating  agreement  for  Items  1 and  2 would  contain  the  following 
Information: 


ITEM  1 

Pass  Fall 


Pass 

ITEM  2 

a - Number  who 
Pass  Item  1 
and 

Pass  Item  2 

b ■ Number  who 
Fall  Item  1 
and 

Pass  Item  2 

c ■ Number  who 

d • Number  who 

Fall 

Pass  Item  1 

Fall  Item  1 

and 

and 

Fall  Item  2 

Fall  Item  2 

AI  ■ (a  X d)  - (b  X c) 


Collect  this  Information  by  referring  to  the  summary  of  test  performance 
and  counting  the  number  of  times  each  of  the  four  events  occurs.  The 
distinction  between  performers  and  non-performers  does  not  apply  for 
this  analysis. 

c.  After  preparing  the  2x2  tables,  apply  the  formula:  Agreement 
Index  - (a  X d)  - (b  x c). 

d.  For  example,  the  2x2  tables  for  the  Items  In  Figure  8-8  are  shown  In 
Figure  8-9.  The  2x2  tables  are  prepared  on  another  form  developed  at  the 
Ordnance  School.  Again  the  minimum  requirement  Is  that  the  Agreement 
Index  be  greater  than  zero.  All  pairs  In  this  sample  SU  meet  that 
requirement . 

e.  If  the  SU  has  more  than  four  Items,  compute  Agreement  Indexes 
between  each  Item  and  overall  performance  on  the  SU.  This  computation 
requires  fonr  steps: 
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• Delete  unacceptable  items  from  total  score*  By 
this  point,  you  may  have  already  found  an  item 
that  must  be  revised  because  80Z  of  the  perform- 
ers do  not  accept  it,  or  because  more  perform- 
ers fail  than  pass  it*  Including  such  items 

in  this  analysis  will  only  distort  the  Agreement 
Indexes* 

• Prepare  a table  indicating  number  of  soldiers 
(performers  and  non-performers  combined)  who 
pass  all  items,  all  but  one  item,  all  but  two, 
and  so  on  down  to  those  who  pass  zero  items* 

The  table  for  Figure  8-10  would  look  like  this: 


No*  Items  passed 

6 

5 

4 

3 

2 

1 0 

No*  of  soldiers 

2 

5 

8 

8 

2 

4 1 

Note:  Results  of 

item 

4 

are 

dropped 

because 

too  many  performers  failed  it  - remem- 
ber the  earlier  requirement  that  more 
performers  must  pass  an  item  than  fall 
it*) 

• Divide  the  sample  into  two  groups  as  close  as 
possible  to  the  same  size*  In  this  example,  the 
groups  split  evenly  between  4 correct  and  3 
correct:  the  IS  soldiers  who  answered  four  or 
more  questions  right  are  the  high  group;  the 

15  soldiers  who  answered  3 or  less  questions 
right  are  the  low  group* 

• Compute  the  Agreement  Index  between  scoring  group 
(high  or  low)  and  performance  on  each  item  (pass 
or  fall)*  Prepare  a 2x2  table  for  each  item  in 
this  way: 

SCORING  GROUP 


High 

Low 

m 

a - Number  of  soldiers 

b ■ Number  of  soldiers 

Pass 

in  high  group  who 

in  low  group  -who 

pass  item 

pass  item 

ITEM 

c - Number  of  soldiers 

d Number  of  soldiers 

Fall 

in  high  group  who 

in  low  group  who 

fall  item 

fall  item 
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8-49.  IS  ITEM-RATING  INDEX  POSITIVE?  If  soldiers  who  rate  themselves  as 
performers  on  a task  do  better  on  an  Item  than  non-performers,  the  Item- 
rating Index  will  be  greater  than  zero.  If  the  Index  Is  zero  or  a nega- 
tive number,  probably  something  Is  wrong  with  the  Item.  S’^ch  Items  must 
be  reviewed  or  revised. 


8-50.  IS  ITEM-SU  INDEX  POSITIVE?  If  high-scoring  soldiers  (high  on  over- 
all SU)  do  better  on  an  Item  than  low  scoring  soldiers  (low  on  overall 
SU),  the  SU-ltem  Index  %rlll  be  greater  than  zero.  If  the  Index  Is  zero 
or  a negative  number,  there  may  be  something  irrong  with  the  Item.  If 
the  Item-rating  Index  Is  not  positive  but  the  SU-ltem  Index  Is  positive, 
review  the  Item  for  possible  revision.  If  neither  Index  Is  positive, 
revise  the  Item.  If  both  Indexes  are  positive  and  the  other  requirements 
are  met,  the  Item  Is  valid. 


8-51.  REVISE  ITEM. 

a.  If  the  people  who  'developed  the  SU  have  followed  development  proce- 
dures, minor  revisions  will  probably  make  the  Items  acceptable.  If  the 
Item  Is  too  hard  (more  performers  fall  than  pass  It),  check  the  wording 

of  the  Item,  check  accuracy  of  the  correct  alternative  and  consider  adding 
Illustrations  to  Increase  clarity. 

If  the  Item  Is  too  easy  (non-performers  pass  It  at  least  as  often  as 
performers),  check  for  unwanted  grammatical  cues,  check  for  cues  In  other 
Items  that  may  give' away  the  Item  In  question,  and  check  possibility  that 
a likely  wrong  alternative  has  been  left  out  of  the  Item. 

b.  If  the  Item  does  not  have  a superficial  deficiency,  you  may  need 
to  make  a major  revision.  First  analyze  responses  of  the  performers  to 
find  which  wrong  alternative  they  usually  select.  Be  sure  that  alternative 
Is  a real-world  alternative  and  Is  wrong. 


Then  check  the  task  analysis  data  to  be  sure  the  behavior  the  Item 
measures  Is  a key  behavior.  If  the  behavior  Is  not  key,  you  may  be  able 
to  coordinate  with  ITED  to  delete  It  from  the  SU. 

If,  after  careful  study,  you  can  find  no  fault  with  the  Item,  and  the 
behavior  Is  key  to  successful  performance  of  the  task,  accept  the  Item 
as  Is.  When  you  decide  whether  to  validate  the  revised  SU.  treat  such 
an  Item  as  though  It  had  been  revised. 


8-52.  HAVE  EXPERTS  REVIEW  REVISED  ITEMS.  Have  at  least  three  acknowl- 
edged experts  compare  the  revised  Item  with  the  original  Item.  Ask  them 
whether  both  Items  have  the  same  content.  Also  have  them  answer  the 
original  and  revised  Items. 


I 
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8-53.  DO  EXPERTS  AGREE  THE  CONTENT  IS  THE  SANE?  By  this  stage  of  the 
validation,  content  of  the  SU  Is  established  and  cannot  be  changed  with- 
out coordinating  with  ITED.  If  any  expert  disagrees  with  the  answer  or 
thinks  the  revision  changed  the  content,  revise  the  Item  again. 


8-54.  ARE  1/3  OF  ITEMS  REVISED?  If  fewer  than  1/3  of  the  Items  In  an 
SU  are  revised,  no  additional  testing  Is  required.  If  1/3  or  more  of 
the  Items  In  an  SU  are  revised,  repeat  the  validation  procedure  for  the 
complete  SU. 


8-55.  SELECT  20  SOLDIERS.  The  only  difference  In  the  procedure  for  a 
revised  SU  Is  that  your  sample  need  contain  only  20  soldiers  with  a range 
of  ability.  Do  not  Include  any  soldiers  who  were  In  the  original  sample 
to  validate  the  revised  SU.  Have  soldiers  rate  their  ability  to  perform 
the  task. 


j 8-56.  AT  LEAST  SEVEN  PERFORMERS  AND  SEVEN  NON -PERFORMERS?  The  sample 

I for  validating  revised  SU,  like  the  original  sample,  should  Include 

I performers  (self-ratings  of  3 or  2)  and  non-performers  (self-ratings  of 

0 or  1).  If  possible  get  10  of  each;  have  at  least  7 of  each.  Perform 
the  rest  of  the  validation  like  you  did  originally. 

I 

! Section  VI.  VALIDATE  WRITTEN  SU  - SUPERVISOR  RATING 


8-57.  INTRODUCTION.  The  supervisor  rating  validation  option  lets  you 
I validate  a written  component  even  though  you  have  access  to  less  than 

I 30  soldiers. 

I This  option  selects  the  sample  by  combining  a supervisor's  rating  of 

a soldier's  observed  or  expected  performance  with  the  soldier's  rating 
I of  his  own  ability  to  perform  a task. 

I 8-58.  CONTACT  SUPERVISOR  WITH  THE  MOST  PEOPLE  IN  THE  MOS.  Since  you 

will  need  several  solalers  In  the  MOS,  It  Is  a good  Idea  to  go  to  the 
supervisor  responsible  for  the  most  soldiers  In  the  NOS  at  the  Instal- 
lation. 


8-59.  HAS  SUPERVISOR  OBSTOVED  SOLDIERS  PERFORM  THE  TASK?  Ask  the 
supervisor  If  he  has  seen  any  of  his  soldiers,  regardless  of  rank,  try 
to  perform  the  task  covered  by  the  SU.  Show  him  the  task  analysis  data 
for  the  task  so  he  knows  the  scope  of  the  task. 
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If  he  has  not  observed  at  least  five  soldiers  perform  the  task,  eventu- 
ally you  will  need  to  contact  another  supervisor,  but  go  ahead  and  collect 
data  for  soldiers  he  has  observed.  If  he  has  not  observed  any  soldiers 
perform  the  task,  tell  him  you  may  need  to  contact  him  later,  but  for 
now  none  of  his  soldiers  are  eligible  to  be  In  the  sample. 

8-60.  COLLECT  RATINGS  ON  TASK.  Have  the  supervisor  answer  one  of  the 
following  questions  for  each  soldier  he  has  seen  perform  the  task.  The 
question  to  ask  depends  on  whether  the  task  noraially  Is  performed  under 
supervision. 

(1)  How  well  can  he  perform  the  task? 

1.  Not  very  well. 

2.  Fairly  well. 

3.  Very  %fell. 

(2)  To  perform  the  complete  task  again,  how  much  technical 

guidance  would  he  require  from  someone  who  knows  how  to 

perform  the  ta^k? 

1.  Lots  of  guidance. 

2.  Some  guidance. 

3.  No  guidance. 


Soldiers  rated 
performers. 


are  performers.  Soldiers  rated  "1"  are  non- 


8-61.  COLLECT  SELF-RATINGS.  Collect  self-ratings  from  each  soldier  the 
supervisor  rates.  When  you  collect  the  self-ratings,  emphasise  the  impor- 
tance of  accurate  self-ratings.  Be  sure  expected. experts  know  that  there 
Is  no  benefit  to  them  or  to  the  SQT  if  they  overrate  their  ability.  Give 
the  soldier  the  task  analysis  data  for  the  task  and  ask  him  one  of  the 
following  questions.  The  question  to  ask  depends  on  tdiether  the  task 
normally  Is  performed  under  supervision. 

(1)  How  well  can  you  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Not  very  well. 

2.  Fairly  well. 

3.  Very  well. 

(2)  To  perform  the  complete  task,  how  much  technical  guidance 
would  you  require  from  someone  who  knows  how  to  perform 
the  task? 


0.  Not  familiar  with  the  task. 

1.  Lots  of  guidance. 

2.  Some  guidance. 

3.  No  guidance. 
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8-62.  FIVE  AGREEMENTS?  Compare  the  supervisor's  rating  of  each  soldier 
with  the  soldier's  self-ratings.  Soldiers  In  the  same  group  for  both 
ratings  are  eligible  to  be  In  Lite  validation  sample.  If  less  than  five 
agreements,  you  will  need  to  get  more  soldiers  before  you  finish  the 
validation,  but  complete  the  rest  of  the  main  strand  (left  side)  of  the 
flow-chart  trhlle  the  soldiers  are  available. 


8-63.  AT  LEAST  TVO  PERFORMERS  AND  TWO  NON -PERFORMERS?  The  five  soldiers 
In  the  validation  sample  must  have  a range  of  ability.  For  this  option, 
there  must  be  at  least  two  performers  and  two  non-performers  among  the 
five.  Otherwise  you  cannot  tell  whether  an  SU  Is  too  easy  or  too  hard. 

For  example.  If  you  have  only  non-performers  in  the  sample  and  they 
all  fall  an  SU,  you  still  do  not  know  whether  performers  could  pass  the 
SU.  If  you  do  not  have  a range  of  ability,  eventually  you  will  need 
to  add  to  the  sample,  but  test  soldiers  you  need  while  they  are  available. 

Do  not  test  more  soldiers  In  one  group  than  you  need.  Save  the  "extra" 
soldiers  In  one  group  than  you  need.  Save  the  "extra"  soldiers  In  case 
the  SU  Is  revised. 


8-64.  ADMINISTER  SU  TO  FIVE  SOLDIERS. 

a.  Although  eventually  you  must  have  five  soldiers  take  each  SU, 
there  are  two  advantages  to  testing  soldiers  as  they  are  available  even 
though  there  are  less  than  five  of  them: 

• You  do  not  have  to  arrange  a time  for  soldiers 
from  different  units  to  get  to  a test  site. 

•You  can  be  alerted  to  faults  in  the  SU  that  will 
require  revision.  You  can  then  revise  the  SU  and 
conserve  remaining  soldiers  for  validating  the 
revision. 

b.  When  you  administer  jthe  SU,  make  conditions  as  similar  as  possible 
to  the  field  conditions.  Use  Illustrations  and  Instruct^lon*  the  same  as 
when  you  test  for  record.  Allow  soldiers  as  much  tlmn  as  they  need  to 
answer  each  question  and  to  review  their  answers. 


8-65.  COMPUTE  RATING-SU  AGREEMENT. 

a.  First,  score  the  tests,  ^ply  the  scoring  rules  developed  for 
each  Item  In  the  SU. 

b.  Next  prepare  a table  that  shows  each  soldier's  rating  (per- 
former or  non-performer)  and  his  performance  on  the  SU  (pass  or  fall). 
Here  Is  what  a table  might  look  like: 
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Soldier 

Rating 

SU  Score 

1 

1* 

1+ 

2 

0** 

1 

3 

0 

0++ 

4 

1 

1 

5 

0 

0 

*1  ■ performer 
**0  ■ non-performer 


-fl  ••  pass 
++0  - fall 


c.  Finally  figure  the  percentage  of  agreement  between  ratings  and  SU 
scores*  First  count  the  number  of  agreements*  An  agreement  Is  when  a 
performer  passes  the  SU  or  a non-performer  falls  the  SU*  In  the  example, 
two  performers  passed  the  SU  and  two  non-performers  failed  It;  therefore 
the  example  has  four  agreements*  Then  divide  the  number  of  agreements  by 
the  number  of  soldiers  In  the  sample*  The  percentage  of  agreement  In  the 
example  Is  80  (4tS)* 


8-66*  IS  THE  AGREEMENT  AT  LEAST  80Z?  A minimum  agreement  of  80  percent 
Is  required  for  the  SU  to  be  acceptable*  With  five  to  nine  soldiers  In 
the  sample,  an  SU  Is  allowed  only  one  disagreement*  If  the  agreement  Is 
80  percent  or  higher,  the  SU  Is  valid*  If  the  agreement  Is  less  than 
80  percent,  you  must  revise  the  SU* 


8-67*  INTERVIEW  SOLDIERS*  If  the  SU  does  not  validate  (less  than  80  per- 
cent agreement).  Interview  each  soldier  one  at  a time  to  Identify  faults 
In  the  SU*  Ask  these  questions: 

(1)  Do  you  think  soldiers  who  answer  these  Items 
correctly  are  able  to  perform  the  task? 

(2)  Does  each  question  relate  to  an  Important  part 
of  doing  the  task? 

(3)  Are  any  Important  parts  of  doing  the  task  left 
out? 

Then  try  to  find  why  a performer  failed  an  Item  or  a non-performer  passed 
the  Item*  That  Information  will  help  you  revise  the  SU* 


I 
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8-68.  REVISE  ITEMS  WITH  DISAGREEMENTS. 

If  performers  do  not  pass  an  item.  It  may  be  too  hard.  Check  wording 
of  the  item,  check  accuracy  of  the  correct  alternative,  consider  adding 
Illustrations  to  Increase  clarity  -^nd  check  the  wrong  alternatives 
performers  choose  to  be  sure  those  alternatives  are  real-world  cholces- 
and  are  really  wrong. 

If  non-performers  pass  an  SU  the  items  may  be  too  easy.  Check  for 
unwanted  grammatical  cues,  check  for  cues  In  other  Items  that  may  give 
away  the  Item  In  question,  and  check  possibility  that  a likely  wrong 
alternative  has  been  left  out  of  the  Item. 


8-69.  ARE  MORE  SOLDIERS  RATED?  This  Is  the  first  question  to  answer 
tfhen  you  validate  a revised  SU.  It  Is  possible  that  you  may  have  some 
soldiers  left  from  the  first  rating  whose  self-rating  agree  with  their 
supervisor's,  but  who  were  not  needed  for  the  first  validation.  Do 
not  Include  soldiers  from  the  first  validation  In  the  sample  for  the 
revised  SU.  The  sample  for  a revised  SU  must  meet  the  original  require- 
ments, that  Is,  at  least  two  performers  and  two  non-performers  among  a 
group  of  five  soldiers. 


8-70.  ARE  MORE  SUPERVISORS  AVAILABLE  TO  RATE  OBSERVED  PERFORMANCE?  If 
you  cannot  meet  requirements  for  the  original  validation  sample  or  for 
validating  the  revised  SU  based  on  observed  performance  from  the  first 
supervisor  you  contact,  find  If  there  are  other  supervisors  of  soldiers 
In  the  MOS  at  the  Installation.  If  there  are,  contact  the  supervisor 
with  the  next  most  In  the  MOS.  If  you  have  exhausted  the  possibilities 
of  observed  performance,  collect  ratings  of  expected  performance. 

8-71.  CONTACT  SUPERVISOR  WITH  NEXT  MOST  IN  MOS.  If  you  have  not  yet 
contacted  all  the  supervisors,  contact  the  remaining  supervisor  who  Is 
responsible  for  the  most  soldiers  In  the  MOS.  Continue  contacting  super- 
visors until  you  meet  the  sample  requirements  or  run  out  of  people  who 
can  rate  on  observations  of  performance. 


8-72.  ARE  THERE  SUPERVISORS  TO  RATE  EXPECTED  PERFORMANCE?  A less 
powerful  but  still  acceptable  basis  for  supervisor's  ratings  Is  expected 
performance.  Contact  the  supervisors  of  soldiers  In  the  MOS  trho  could 
not  be  rated  on  observed  performance.  Usually  you  will  simple  re-contact 
each  supervisor. 


8-73.  HAS  SUPERVISOR  BEEN  IN  THE  SLOT  FOR  90  DATS?  The  first  require- 
ment for  expected  performance  ratings  Is  that  the  rater  be  an  experienced 
supervisor.  The  minimum  level  of  experience  is  90  days.  If  the  super- 
visor has  not  been  In  his  slot  for  90  days,  contact  another  supervisor. 
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8-74.  COLLECT  RATING  FOR  EACH  SOLDIER  WHO  HAS  BEEN  IN  THE  UNIT  30  CAYS. 

a.  The  second  requirement  for  expected  performance  ratings  Is  that 
the  supervisor  Is  familiar  with  the  overall  ability  of  soldiers  he  rates. 

If  the  soldier  has  been  under  the  rater's  supervision  for  30  days,  this 
requirement  Is  met. 

b.  Give  the  supervisor  the  task  analysis  data  for  the  task  and  have 
him  answer  this  question  for  each  soldier  eligible  to  be  in  the  valida- 
tion sample: 

(1)  How  confident  are  you  that  he  can  do  this  task? 

0.  Sure  he  cannot  do  It. 

1.  Doubt  he  can  do  It. 

2.  Think  he  can  do  It. 

3.  Sure  he  can  do  It. 

Soldiers  rated  0 or  1 are  non-performers.  Soldiers  rated  2 or  3 are 

performers.  I 

8-75.  COLLECT  SELF-RATINGS. 

a.  Collect  self-ratings  from  each  soldier  Che  supervisor  rates.  Give 
the  soldier  the  task  analysis  data  for  the  task  and  ask  him  one  of  the 
following  questions.  The  question  Co  ask  depends  on  whether  the  task 

normally  Is  performed  under  supervision.  i 

(1)  How  well  can  you  perform  the  task? 

■ 

0.  Not  familiar  with  the  task. 

1.  Not  very  well. 

2.  Fairly  well. 

3.  Very  well.  ; 

(2)  To  perform  Che  complete  task,  how  much  technical  guidance 
would  you  require  from  someone  who  knows  how  to  perform  the 
task? 

0.  Not  familiar  with  the  cask. 

1.  Lots  of  guidance. 

2.  Some  guidance.  I 

3.  No  guidance. 

Soldiers  who  choose  "0"  or  "1"  are  non-performers;  soldiers  who  choose 
"2"  or  "3"  are  performers.  ' 

b.  The  remaining  procedure  for  validating  an  SU  using  supervisor's 
ratings  of  expected  performance  is  the  same  as  for  ratings  of  observed 
performance. 
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Section  VII.  VALIDATE  WRITTEN  SU  - PANEL-OF-EXPERTS 


8-76.  INTRODUCTION.  This  option  lets  you  valldste  an  SU  with  fewer 
soldiers  than  the  self-rating  option  even  though  supervisors  of  the 
soldiers  are  not  available.  With  this  option,  you  select  the  validation 
sample  by  combining  the  evaluation  by  a panel  of  experts  and  self-ratings 
made  by  5 or  10  soldiers. 


8-77.  SELECT  PANEL  OF  EXPERTS. 

a.  The  panel  should  Include  three  people  who  know  a lot  about  the 
task  covered  by  the  SU  and  can  do  the  task  well.  TVo  sources  of  experts 
are  senior  NCO's  and  Instructors  in  the  MOS.  If  you  doubt  the  qualifi- 
cations of  a potential  expert.  Interview  his  peers  and  supervisor.  If 
they  say  the  expected  expert  can  do  the  specific  task  very  well,  consider 
Including  him  on  the  panel.  Otherwise  do  not  Include  him  on  the  panel. 
Test  developers  ahould  not  be  on  the  panel. 

b.  After  you  locate  several  people  you  expect  to  be  experts,  give 
them  the  task  analysis  data  for  the  SU  and  have  them  answer  one  of  these 
questions.  When  you  collect  the  self-ratings,  emphasize  the  Importance 
of  accurate  self-ratings.  Be  sure  expected  experts  know  that  there  Is 
no  benefit  to  them  or  the  SQT  If  they  overrate  their  ability.  The 
question  to  ask  depends  on  whether  the  task  normally  Is  performed  under 
supervision. 

(1)  How  well  can  you  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Not  very  well. 

2.  Filrly  well. 

3.  Very  well. 

(2)  To  perform  the  complete  task,  how  much  technical 

guidance  would  you  require  from  someone  who  knows 

how  to  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Lots  of  guidance. 

2.  Some  guidance. 

3.  No  guidance. 

Per  e who  choose  number  3,  "Very  well"  or  "No  guidance"  are  eligible  to 
sc.Vw  on  the  panel. 
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8-78.  SELECT  SOLDIERS. 

a.  Soldiers  for  this  option  may  be  Incumbents  In  the  MOS,  or  AIT 
students,  or  a combination  of  Incumbents  and  AIT  students.  Incumbents 
In  the  MOS  do  not  have  to  be  at  the  skill  level  for  the  SQT,  In  fact, 
a range  of  skill  levels  la  desirable  since  that  helps  assure  a spread 
of  ability  to  perform  the  tasks. 

Although  the  final  aam^le  requires  only  5 soldiers,  try  to  have  at 
least  10  available  because  It  la  rare  when  everyone  eligible  meets 
requirements  for  the  sample. 

b.  Give  the  soldiers  the  task  analysis  data  for  the  SU  and  have  them 
answer  one  of  the  questions.  When  you  collect  self-ratings,  emphasize  the 
Importance  of  accuracy.  Be  sure  expected  experts  know  there  Is  no  bene- 
fit to  them  or  the  SQT  If  they  overrate  their  ability.  The  question  to 
ask  depends  on  whether  the  task  normally  Is  performed  under  supervision. 

(1)  How  well  can  you  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Not  very  well. 

2.  Fairly  well. 

3.  Very  well. 

(2)  To  perform  the  complete  task,  how  much  technical 

guidance  would  you  require  from  someone  who  knows 

how  to  perform  the  task? 

0.  Not  familiar  with  the  task. 

1.  Lots  of  guidance. 

2.  Some  guidance. 

3.  No  guidance. 

Soldiers  who  rate  themselves  low  (0  or  1)  are  In  the  non-performer  group; 
soldiers  who  rate  themselves  high  (2  or  3)  are  In  the  performer  group.  A 
soldier  may  be  In  the  performer  group  for  one  task  and  In  the  non-performer 
group  for  another  task. 


8-79.  HAVE  EXPERTS  RATE  SOLDIERS. 

a.  The  panel  trill  Interview  each  soldier  to  determine  whether  the 
soldier  Is  a performer  or  non-performer  on  each  task.  The  panel  will 
develop  their  own  Interview  procedures  and  questions  based  on  the  task 
analysis  data.  The  panel  Is  free  to  ask  any  questions  thay  desire  about 
task  performance.  The  panel  can  ask  for  locations.  Identifications, 
procedures,  special  techniques  and  tools.  The  members  should  probe 
deeply  by  asking  follow-up  questions  based  on  a soldier's  responses. 
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b.  Before  the  panel  Interviews  anyone  for  record,  have  them  rehearse 
their  procedure.  If  you  are  short  of  soldiers,  have  members  of  the  TDA 
role-play  as  potential  subjects.  Monitor  the  rehearsal  to  be  sure  that 
the  panel  does  not  appear  threatening  to  soldiers  It  Interviews. 

c.  Have  the  panel  Interview  one  soldier  at  a time.  Each  expert  on 
the  panel  will  classify  each  soldier  as  either  a performer  or  a non-^ 
performer.  In  case  of  disagreement  among  the  three  experts,  the  rating 
that  two  experts  agree  on  will  be  accepted. 

8-80.  AT  LEAST  FIVE  EXPERT  - SELF-RATING  AGREEMENTS?  Compare  the 
experts'  rating  of  each  soldier  with  the  soldier's  self-rating.  Soldiers 
who  are  in  the  same  group  (performer  or  non-performer)  for  both  ratings 
can  be  In  the  validation  sample.  If  less  than  five  agreements,  get 
more  soldiers. 


8-81.  AT  LEAST  TWO  PERFORMERS  AND  TWO  NONPERFORMERS?  The  validation 
sample  must  Include  a range  of  ability.  Otherwise,  you  cannot  tell 
%diether  an  SU  Is  too  easy  or  too  hard.  For  example.  If  you  have  only 
non-performers  In  the  sample  and  they  all  fall  the  SU,  you  still  do 
not  know  whether  performers  could  pass  the  SU.  The  minimum  requirement 
Is  two  In  each  group. 


8-82.  SELECT  MORE  SOLDIERS.  If  you  have  less  than  two  performers  and 
two  non-performers,  add  soldiers,  collect  self-ratings  and  have  experts 
rate  soldiers  until  you  have  at  least  two  soldiers  per  group  and  a total 
of  five. 


8-83.  ADMINISTER  SU  TO  FIVE  SOLDIERS.  When  you  administer  the  SU,  make 
the  conditions  as  similar  as  possible  to  the  field  conditions.  Use  Illus- 
trations and  Instructions  the  sane  as  when  you  test  for  record.  Allow 
soldiers  as  much  time  as  they  need  to  answer  each  question  and  to  review 
the  answers. 


8-84.  COMPUTE  RATING-SU  AGREEMENT. 

a.  First,  score  the  tests.  Apply  the  scoring  rules  developed  for 
each  Item  In  the  SU. 

b.  Next  prepare  a table  that  shows  each  soldier's  rating  (performer 
or  non-performer)  and  his  performance  on  the  SU  (pass  or  fall).  Here  Is 
what  a table  might  look  like: 
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Subject 

Rating 

SU  Score 

1 

1* 

1+ 

2 

0** 

1 

3 

0 

0++ 

4 

1 

1 

5 

0 

0 

*perforiner  -fpass 

**nonper former  -H-fail 


c.  Finally  figure  the  percentage  of  agreement  between  ratings  and  SU 
scores.  First  count  the  number  of  agreements.  An  agreement  Is  when  a 
performer  passes  the  SU  or  a non-performer  falls  the  SU.  With  the  example, 
two  performers  passed  the  SU  and  two  non-performers  failed  it;  therefore 
the  example  has  four  agreements.  Then  divide  number  of  agreements  by 
number  of  soldiers  In  the  sample.  For  the  example,  the  percentage  of 
agreements  Is  80  (4  7 5). 


8-85.  IS  THE  AGREEMENT  AT  LEAST  80Z?  A minimum  agreement  of  80  percent 
Is  required  for  the  SU  to  be  acceptable.  With  five  to  nine  soldiers  in 
the  sample,  an  SU  Is  allowed  only  one  disagreement.  If  the  agreement  is 
80  percent  or  higher,  the  SU  Is  valid.  If  the  agreement  Is  less  than  80 
percent,  revise  the  SU. 


t 


j 


* 


8-86.  INTERVIEW  SOLDIERS.  If  the  SU  does  not  validate  (less  than  80  per 
cent  agreement).  Interview  each  soldier  one  at  a time  to  identify  faults 
In  the  SU.  Ask  these  questions: 

• Do  you  think  soldiers  who  answer  these  Items 
correctly  are  able  to  perform  the  task? 

• Does  each  question  relate  to  an  Important  part 
of  doing  the  task? 

• Are  any  important  parts  of  doing  the  task  left  out? 

Try  to  find  why  a performer  failed  an  Item  or  a non-performer  passed  the 
Item.  That  Information  will  help  when  you  revise  the  SU. 


■a' 
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8-87.  REVISE  ITEMS  WITH  DISAGREEMENTS.  If  performers  do  not  pass  an 
Item,  It  may  be  too  hard.  Check  trardlng  of  the  Item,  check  accuracy  of 
the  correct  alternative,  consider  adding  Illustrations  to  Increase 
clarity  and  check  the  wrong  alternatives  performers  choose  to  assure 
those  alternatives  are  real-world  choices  and  really  wrong.  If  non- 
performers pass  an  SU,  the  Items  may  be  too  easy.  Check  for  unwanted 
grammatical  cues,  check  for  cues  In  other  Items  that  may  give  away  the 
Item  In  question  and  check  possibility  that  likely  wrong  alternatives 
have  been  left  out  of  an  Item. 


8-88.  .lAVE  EXPERTS  REVIEW  REVISED  ITEMS.  Have  at  least  three  acknowl- 
edged subject  matter  experts  compare  the  revised  Item  with  the  original 
Item.  Ask  them  whether  both  items  have  the  same  content.  Also  have  them 
answer  the  original  and  revised  Items. 


8-89.  ’ ' EXPERTS  AGREE  THE  CONTENT  IS  THE  SAME? 

a.  By  this  stage  of  the  validation,  content  of  the  SU  Is  established 
and  cannot  be  changed  without  coordinating  with  ITED.  If  any  expert  dis- 
agrees with  the  answer  or  thinks  the  revision  changed  the  content,  revise 
the  Item  again. 

b.  If  experts  agree  the  content  Is  the  same,  validate  the  revised  SU. 
Perform  the  same  procedure  described  for  the  original  validation. 
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CHAPTER  9 

DEVELOP  PERFORMANCE  CERTIFICATION  (PCC)  SU 
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9-1.  INTRODUCTION. 

a.  The  allocation  of  taska  to  the  Perfornance  Certification  Compo- 
nent (PCC)  IMS  discussed  In  chapter  4.  Despite  the  serious  weaknesses 
of  the  PCC  as  a seans  of  measuring  a soldier’s  proficiency.  It  appears 
the  only  iMy  that  cettaln  highly  akllled  performances  can  be  evaluated. 

Performance  Certification  is  the  component  where  you  should  consider 
testing  a task  that 

(a)  cannot  be  validly  tested  In  the  WC  of  Its  skilled  hands-on 
requirements,  and 

(b)  cannot  be  tested  In  the  HOC  because  of  administrative 
constraints. 

Sometimes  a hands-on  skill  simply  takes  too  long  to  demonstrate,  or 
requires  equipment,  terrain  or  other  conditions  that  prevent  Its  testing 
In  the  HOC. 

Cutting  a roadbed  with  a bulldozer  Is  a good  example.  To  test  this 
task  In  the  HOC  wciild  probably  take  several  hours  and  a sizeable  expanse 
of  terrain.  Moreover,  equipment  and  fuel  requirements  probably  could 
not  be  Justified  merely  for  testing  purposes. 

b.  The  thinking  behind  the  PCC  Is  that  such  tasks  can  be  "tested"  on 
the  Job.  By  moving  the  test  to  the  Job  site,  we  can  have  the  soldiers' 
supervisor  serve  as  scorer  in  evaluating  the  soldier's  performance  during 
the  normal  course  of  his  Job  duties. 

Some  natural  PCC  already  exist.  Anmal  rifle  qualification  Is  one  such 
example,  where  a soldier's  task  proficiency  Is  evaluated  In  the  field, 
but  under  conditions  essentially  the  same  as  one  would  set  up  for  a hands- 
on  test.  And  this  task  Is  being  Included  In  the  PCC  for  relevant  NOS. 

Host  tasks  that  are  candidates  for  the  PCC,  however,  will  not  be 
structured  already  for  testing  In  a field  unit,  so  the  test  developer 
must  prepare  a scoreshcet  and  carefully  phrased  Instructions  for  the 
evaluator  (supervisor)  on  how,  when,  where  and  under  what  conditions  a 
soldier's  task  performance  Is  to  be  scored. 
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c.  Before  discussing  procedure  for  developing  a PCC  scorable  unit, 
weaknesses  Inherent  In  the  PCC  should  be  noted  carefully.  These  weak- 
nesses are  of  two  general  types  - both  Involving  probleas  of  standardi- 
zation. The  first  pertains  to  difficulties  In  achieving  standard 
conditions  for  task  performance  for  all  soldiers.  In  setting  up  a test, 
as  was  done  In  the  HOC,  we  took  great  care  In  Insuring  that  conditions 
were  the  sane  for  each  soldier  going  through  the  test  station.  In  the 
PCC,  where  a soldier's  task  performance  will  be  scored  as  part  of  his 
normal  Job  duties.  It  will  be  nearly  Impossible  to  control  the  equality 
of  conditions  from  man  to  man,  location  to  location.  A second  weakness 
In  the  PCC  Is  that  the  scorer  (supervisor),  who  Is  beyond  control  of  a 
Test  Control  Officer,  In  most  cases  will  not  be  as  well  trained  or  as 
motivated  to  do  a conscientious  and  unbiased  job  of  scoring  as  a scorer 
in  the  HOC.  The  supervisor  probably  will  have  formed  likes  and  dislikes 
among  his  subordinates.  Just  as  serious  Is  the  possibility  that  a super- 
visor won't  go  to  the  trouble  to  score  task  performance  by  carefully 
controlled  observation,  but  simply  sit  In  his  office  and  fill  out  the 
scoiesheets  on  the  basis  of  his  preconceived  notions  of  how  the  scores 
should  come  out . 

d.  These  shortcomings  of  the  PCC  cannot  be  overcome  entirely,  which 
Is  why  a task  should  be  assigned  to  the  PCC  only  as  a last  resort. 

But,  where  a PCC  Is  necessary,  one  should  make  every  effort  to  (a) 
define  the  limits  of  variation  In  performance  conditions  that  can  be 
tolerated,  and  (b)  make  the  supervisor's  scoring  requirements  as  simple, 
objective,  and  feasible  as  possible. 

e.  Keep  In  mind  that  the  PCC,  with  Its  Inherent  problems  In  admin- 
istration and  support  may  not  be  tested  In  all  locations.  While  every 
effort  should  be  made  to  select  Items  that  are  widely  testable,  the.’e  Is 
no  requirement  that  all  soldiers  take  the  PCC.  Those  soldiers  that  do 
not  will  receive  an  "N"  score  and  will  not  be  penalized  for  not  having 
taken  the  PCC. 


f.  There  Is  no  requirement  that  the  PCC  be  administered  only  once 
to  a soldier.  In  fact,  soldiers  who  fall  an  Initial  PCC  should  be 
allowed  to  be  evaluated  repeatedly  until  a passing  score  Is  obtained 
or  It  Is  evident  that  their  score  reflects  their  actual  capability. 
This  repetition  will  not  be  feasible  In  all  cases  but  where  It  Is,  It 
should  be  encouraged. 


i 9-2.  REVIEW  TASK  ANALYSIS.  Work  begins  with  a careful  review  of 

the  task  analysis  (see  Chapter  3)  Just  as  It  does  in  developing  an 

!SU  for  the  HOC  or  the  WC.  In  fact,  the  procedure  for  developing  a 

scorable  unit  for  the  PCC  Is  essentially  the  same 'as  developing  a SU 
for  the  Hands-On  Component. 
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The  major  difference  la  that  the  teat  conditions  and  evaluator'a 
perspective  can  vary  more  than  la  acceptable  for  the  HOC.  Therefore, 
when  you  develop  a PCC  scorable  unit,  you  must  prepare  detailed  guidance 
for  evaluating  the  task  under  the  various  acceptable  conditions. 


9-3.  ARE  CONDITIONS  STANDARD  ON  THE  JOB? 

a.  The  first  step  Is  to  specify  acceptable  conditions.  Sometimes, 
essential  conditions  on  the  job  will  be  the  same.  The  mile  run  on  the 
APEE  will  be  essentially  the  same  everywhere.  But  the  PCC  can  still  be 
valid  even  If  the  conditions  vary  on  the  job. 

If  conditions  are  not  always  the  same,  describe  each  aspect  of  the  job 
condition  that  must  be  present  before  job  performance  Is  rated.  Consider 
the  amount  of  behavior,  e.g.,  grade  100  yards  of  roadbed;  the  quality  of 
the  materials  worked  with,  e.g.,  shale,  sand,  tundra;  weather  conditions 
(rain,  snow,  heat);  and  amount  of  supervision. 

You  may  end  up  with  several  mixes  of  aspects  of  the  job  conditions. 

It  Is  reasonable  to  require  less  behavior  and  allow  more  supervision  when 
materials  worked  with  and  weather  are  more  "difficult"  than  normal. 

b.  Review  descriptions  of  conditions  with  a subject  matter  expert. 

The  review  should  have  two  goals: 

(1)  Be  sure  the  conditions  and  standards  are  made 
as  equivalent  as  possible. 

(2)  Be  sure  you  Include  all  conditions  under  which 
competent  Incumbents  could  be  expected  to 
perform  the  task. 


9-4.  DOES  TASK  RESULT  IN  A PRODUCT? 

a.  The  second  step  In  developing  a PCC  scorable  unit  Is  to  decide 
how  to  evaluate  a soldier's  proficiency  under  each  set  of  acceptable 
conditions.  The  method  of  evaluation  depends  first  on  whether  the  task 
results  In  a product.  If  It  does,  you  must  decide  whether  the  commander 
will  have  or  can  find  an  existing  representative  product  for  each  soldier 
who  has  performed  the  task  during  the  preceding  year.  If  representative 
products  are  available,  the  commander  should  evaluate  the  products. 
Prepare  a scoresheet  supplemented  by  a template.  If  possible,  that  will 
describe  an  acceptable  product  for  each  condition.  The  procedure  for 
developing  a product-scored  SU  for  the  HOC  should  be  followed  here. 

b.  If  the  task  results  In  a product  that  may  not  be  available  for 
scoring,  or  If  -the  task  requires  process  measurement,  prepare  a score- 
sheet  for  the  commander  or  his  representative  to  observe  the  performance 
of  the  task. 
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If  the  procedure  varies  under  sone  conditions,  prepare  a separate 
scoresheet  for  each  set  of  conditions  which  causes  the  procedure  to 
vary.  In  developing  a process-scored  SU  for  the  PCC,  the  corresponding 
guidance  for  a process-scored  HOC  should  be  followed. 

c.  A sample  process-scored  SU  with  scoring  Instructions  Is  shown  at 
Appendix  A. 


9-5.  DEVELOP  PROCEDURES  TO  ADMINISTER  SU.  After  you  prepare  guidance 
and  materials  for  scoring  an  SU,  you  must  develop  procedures  to  adminis- 
ter the  SU.  These  procedures  should  answer  five  questions: 

(1)  How  many  trials  will  each  examinee  be  allowed 
to  demonstrate  mastery? 

(2)  If  more  than  one  trl/;!  Is  allowable,  how  much 
time  should  be  betveen  trials? 

(3)  Who  (specific  duty  position)  %rlll  certify  mastery, 
that  Is,  score  the  test? 

(4)  Who  %d.ll  fill  out  the  mark  sense  form? 

(5)  How  will  each  soldier  know  he  Is  being  tested? 

Submit  the  procedures  for  administering  the  SU  to  ITED  at  the  same  time 

you  submit  the  PCC.  These  materials  should  accompany  the  SQT  sample. 


9-6.  DEVELOP  PLAN  TO  VALIDATE  PCC. 

a.  Every  SU  in  the  SQT  must  be  validated.  This  requirement  applies  ! 

as  much  to  the  PCC  as  to  any  other  component.  Some  tasks  may  be  In  the  I 

PCC  because  there  Is  not  enough  equipment  to  administer  a hands-on  SU  In  | 

the  unit,  but  there  Is  enough  equipment  to  test  five  soldiers  on  the  SU. 

In  such  cases,  follow  the  procedure  In  chapter  6.  ^ 

b.  If  you  cannot  validate  the  SU  by  following  the  procedures  for  the  i 

HOC,  you  must  develop  another  validation  plan.  At  a minimum,  your  valid-  i 

atlon  plan  should  call  for  submitting  the  PCC  materials  to  at  least  three  ■ 

commanders  and  five  representative  Incumbents.  Interview  the  commanders 

and  Incumbents  to  answer  the  following  questions: 

(1)  Are  task  boundaries  accurate? 

(2)  Is  task  critical  at  MOS  skill  level? 

(3)  Can  soldiers  be  observed  doing  task?  (Commanders  only.) 

(4)  Can  scoring  be  standard  In  specific  unit? 

(Commanders  only.) 
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c.  Submit  the  plan  (not  the  results)  for  validating  each  SU  In  the 
component  to  ITED  at  the  same  time  you  submit  the  PCC<  These  materials 

I should  accompany  the  SQT  sample. 

I 9-7.  DEVELOP  PLAN  TO  MONITOR  PCC. 

a.  Because  test  conditions  for  the  PCC  are  so  variable,  the  TDA 
must  propose  a procedure  for  monitoring  the  PCC.  This  procedure  should 
protect  against  two  unlikely,  but  possible,  abuses. 

The  first  abuse  Is  that  a commander  may  "protect"  the  unit  by  certi- 
fying everyone  In  the  unit  as  a GO.  The  second  abuse  Is  that  a commander 
may  evade  the  requirement  entirely  by  marking  everyone  as  Not  Rated  even 
though  the  task  Is  routinely  performed. 

b.  The  recommended  procedure  for  monitoring  a PCC  Is  to  spot-check 
ability  of  soldiers  In  some  of  the  units.  The  units  checked  should  be 
those  that  appear  to  have  an  unreasonable  number  of  "GO”  or  "Not  Rated" 
scores  plus  a random  sample  of  other  units. 

c.  The  plan  for  Implementing  the  recommended  procedure  for  monitoring 
the  PCC  should  contain  three  parts.  In  the  first  part,  specify  the 

J criteria  for  identifying  units  to  be  checked.  For  example,  you  may  be 

confident  that  at  least  one-third  of  a unit's  maintenance  nersonnel  will 
change  a gun  tube  during  the  test  period. 

A higher  "Not  Rated"  level  will  alert  you  to  a likely  abuse.  In 
the  second  part,  specify  how  you  propose  to  check  the  units.  Usually 
the  check  %rlll  be  a small  scale  performance  test.  But  also  consider 
approaches  such  as  Interviewing  Incumbents  or  checking  maintenance 
records. 

In  the  third  part  of  the  plan  tell  who  will  do  the  check.  Probably 
the  most  common  proposal  will  be  to  send  a team  from  the  TDA.  But  you 
should  also  consider  approaches  such  as  working  through  the  Maintenance 
Assistance  Instruction  Team  at  an  Installation. 

d.  Submit  the  plan  for  monitoring  the  PCC  to  ITED  at  the  sasm  time 
you  submit  the  PCC.  These  materials  should  accompany  the  SC^T  sample. 


1 


i 


Figure  10-1.  Prepare  SQT  Notice 
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CHAPTER  10 
PREPARE  SQT  NOTICE 


10-1.  INTRODUCTION.  The  purpose  of  the  SQT  Notice  Is  to  tell  the 
soldier  precisely  on  what  he  will  be  tested.  Unlike  previous  MOS  tests, 
the  soldier  will  be  told  exactly  what  he  must  know  and  be  able  to  do  to 
pass  the  test.  For  the  HOC  and  PCC,  this  Is  accomplished  by  providing  a 
copy  of  the  scoresheet  for  each  SU  In  the  notice.  Descriptions  of  writ- 
ten SUs  are  more  difficult  to  prepare  since  we  want  the  soldier  to  know 
what  he  will  be  asked  to  do  without  actually  giving  away  the  answers. 
How  to  prepare  an  entry  for  the  Notice  on  a written  SU  Is  discussed  In 
some  detail.  In  addition,  a description  of  the  overall  format  for  the 
Notice,  a checklist  for  written  SU  entries,  and  a sample  SQT  Notice  are 
given  In  Appendix  B. 

10-2.  EXTRACT  HOC  SCORESHEET.  The  SQT  Notice  for  the  HOC  will  consist 
of  an  extract  of  the  Instructions  as  they  are  to  be  read  to  the  examinee 
and  the  performance  measures  as  they  appear  on  the  scoresheet.  This 
Information  Is  extracted  verbatim  for  the  SQT  Notice. 

Be  sure  no  special  instructions  to  the  scorer  were  Included  on  the 
scoresheet  that  should  not  have  been,  e.g.,  the  equipment  fault  to  be 
Induced  In  a trouble-shooting  task.  Such  Information  should  have  been 
contained  In  Instructions  to  the  scorer,  separately  from  the  scoresheet. 

Compromising  Information  of  this  sort  must  be  deleted  from  the  Notice 
and,  for  that  matter,  from  the  scoresheet. 


10-3.  EXTRACT  PCC  SCORESHEET.  The  Notice  entry  for  a PCC  SU  Is  prepared 
exactly  the  same  as  for  the  HOC.  Preparation  Is  essentially  a matter  of 
reproducing  the  appropriate  scoresheet. 


10-4.  PREPARE  HC  SU  DESCRIPTION. 

a.  Description.  The  entry  in  the  Notice  for  a written  SU  consists 
of  a short  descriptive  paragraph  summarizing  what  the  soldier  will  be 
asked  to  do,  what  he  will  be  given  to  do  It  with,  how  he  will  be  expected 
to  respond,  and  what  Is  required  to  pass. 

b.  Specify  Task  Coverage.  It  may  be  necessary  to  tell  the  soldier 
what  aspect  or  part  of  the  task  Is  to  be  covered  In  the  SU.  In  most 
cases.  It  will  be  evident  to  the  soldier  from  the  task  title  and  the 
description  of  response  conditions  and  requirements  that  you  will  provide. 
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c.  Specify  Response  Conditions*  State  everything  the  soldier  will 
be  given  In  the  SU:  how  the  problem,  situation  or  questions  will  be 
framed;  media  used  (photos,  tables,  charts,  etc.);  and  any  tools  or 
aids  he  will  be  using. 

For  example:  "You  will  be  given  a map  protractor,  grid  scales,  and 
selected  parts  of  maps...";  or,  "You  will  be  given  a partially  completed 
DA  Form  2400  (Equipment  Utilization  Record)  and  a blank  DA  Form  2404 
(Equipment  Inspection/Maintenance  Worksheet)." 

d.  Specify  Response  Requirements.  Next,  tell  the  soldier  what  spe- 
cific task  actions  or  products  he  will  be  performing  or  evaluating. 

For  example:  "You  must  carry  out  the  necessary  calculations  and 
conversions  asked  for...";  "You  will  be  asked  to  Identify  where  entries 
are  made  on  DA  Form  2400  and  2404...";  or,  "You  must  match  the  correct 
shape  and  color  with  the  words  that  complete  the  [NBC]  marker." 

e.  Specify  Response  Mode.  Next,  If  not  ilready  evident  from  the 
foregoing  statement  of  response  requirements,  tell  the  soldier  how  he 
will  be  responding.  In  most  cases,  this  will  be  simply  a matter  of 
/laying,  "You  must  select  the  correct.. . [action]  [treatment]  [picture] 
[sequence] [answer] .. .from  those  listed." 

f.  Specify  Passing  Standard.  Finally,  tell  the  soldier  the  number 
of  Items  In  the  SU  and  the  number  he  must  pass  to  receive  a "CO."  For 
example,  "This  unit  will  consist  of  five  questions.  You  must  answer 
four  questions  correctly  to  score  a "GO"  for  the  unit." 


g.  Select  Sample  Items.  Even  a detailed  description  of  a written  SU 
may  not  tell  the  soldier  exactly  what  he  must  know  and  be  able  to  do  to 
pass  the  SU.  This  Is  especially  true  of  performance-based  SU. 

If  the  description  leaves  any  doubt  about  the  requirements  to  pass  the 
SU,  add  a sample  Item  to  the  description.  If  the  Item  requires  an 
Illustration  for  the  SQT,  Include  the  Illustration  In  the  description. 
Generally,  consider  Including  a representative  Item  In  the  SQT  Notice 
for  every  performance-based  SU. 


10-5.  VERIFY  TASK  NUMBERS  AND  TITLES  WITH  SM.  Check  to  be  sure  that 
the  title  and  number  of  each  task  In  the  Notice  a.e  exactly  the  same 
as  given  In  the  SM. 


10-6.  ANY  CHANGES  TO  MATERIAL  IN  SM?  During  SQT  development  you  may 
have  corrected  a significant  error  In  a task  analysis,  divided  a large 
task  Into  subtasks,  or  even  discovered  a critical  task  that  somehow 
failed  to  appear  In  the  SM.  If  any  changes  of  this  sort  affect  Items 
that  appear  In  the  SQT,  write  then  up  and  distribute  then  with  the  S()T 
Notice. 
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APPENDIX  A 


HEADQUARTERS, 
DEPARTMENT  OF  THE  ARMY 
WASHINGTON,  DC,  20310 


: PERFORMANCE  CERTIFICATION  COMPONENT 

i SQT  2,  MOSC  16J10 

i ' SCORING  INSTRUCTIONS  TO  SUPERVISORS 


1«  A performance  certification  component  is  included  in  the  SQT  in  order 
to  determine  whether  soldiers  have  certain  skills  and  abilities  which  are 
important  but  which  cannot  be  tested.  For  example,  in  the  SQT  for  Engi- 
neer Heavy  Equipment  Operator,  it  is  not  possible  to  determine  from  a 
written  test  whether  a soldier  has  the  ability  to  grade  a roadbed.  It 
is  not  practical  to  set  up  a hands-on  test  to  measure  this  ability  but 
soldiers  who  perform  the  task  on  the  Job  can  be  observed  by  a supervisor 
and  certified  as  able  (or  not  able)  to  perform  the  task  satisfactorily. 

2.  The  supervisor  who  observes  the  soldier's  performance  on  the  tasks 
in  the  performance  certification  is  responsible  for  scoring  the  soldier. 

The  scoring  must  be  done  by  a supervisor  in  the  grade  of  E-6  or  above. 

In  order  to  certify  the  soldier's  performance,  performance  of  the  task 
must  have  been  observed  within  the  12-month  period  preceding  the 
beginning  of  the  quarter  in  which  the  SQT  is  scheduled.  The  supervisor 
must  have  clear  knowledge  of  the  soldier's  performance.  Soldiers  per- 
forming the  task  during  the  preceding  12  months  who  can  be  scored  by  a 
supervisor  will  be  scored  in  accordance  with  the  standards  on  the 
Performance  Certification  Scoresheet.  If  the  task  was  not  performed 
during  the  required  time  period  or  if  the  supervisor  does  not  have  a 
clear  knowledge  of  the  soldier's  performance,  then  the  soldier  is  to  be 

scored  "N"  (not  observed).  The  soldier  will  not  be  penalized  for  a i 

score  of  "N." 

3.  a.  SQT  2,  MOSC  16J10  consist  of  a written  component  and  a per- 
formance certification  component. 


b.  The  performance  certification  component  consists  of  one  event. 


TASK  NUMBER 

PAGE 

SOLDIER'S  MANUAL 

NUMBER 

TASK 

441-16J-1039 

1-5 

ENERGIZE  THE  ELECTRONIC  SEARCH 
CENTRAL 

4.  A scoresheet  page  must  be  completed  for  each  soldier  holding  MOSC 
16J10.  Print  the  soldier's  last  name,  first  name,  and  middle  initial. 
Enter  the  nine-digit  SSN  as  it  appears  on  the  soldier's  social  security 
or  identification  card.  Enter  the  TCO  nu^er  which  is  to  be  provided 
by  the  TCO.  The  S()T  nuiri)er  for  soldiers  holding  MOSC  16J10  is  16J2476. 

A-I 
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APPENDIX  A (continued) 

5.  The  Information  on  the  scoresheet  Is  to  be  transcribed  to  an  attached 
mark  sense  answer  sheet  which  must  be  returned  to  the  TCO  not  later  than 

(date) • The  performance  certification  component  scoresheet  Is 

to  be  given  to  the  soldier* 

6.  In  transcribing  the  Identifying  Information  to  the  mark  sense  answer 
sheet,  print  the  soldier's  name  on  the  mark  sense  sheet  exactly  as  It 
appears  on  the  scoresheet  and  then  under  each  letter  blacken  the  space 
corresponding  to  that  letter.  Print  the  SSN  on  the  mark  sense  form  and 
then  blacken  the  spaces  corresponding  to  the  numbers.  Follow  the  same 
procedure  In  transcribing  the  TCO  number  and  the  test  number  to  the 
mark  sense  form. 

7.  The  test  scores  are  to  be  transcribed  from  the  scoresheet  to  the 
mark  sense  answer  sheets  as  follows: 

a.  The  scores  for  Scorable  Unit  1 are  to  be  entered  on  the  Mark 
Sense  answer  sheet  at  Unit  1.  If  the  event  was  not  certified, 
blacken  the  "N"  block  with  a not  too  soft  lead  pencil. 


N 0 1 


1 ■ □ □ 

2 □ □ □ 


b.  If  the  soldier  does  not  qualify  on  the  scorable  unit,  blacken 
the  0 block. 


c.  If  the  soldier  qualifies  on  the  scorable  unit,  blacken  the  1 block. 


f 

, I 
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SOLDIER'S  NAME 


APPENDIX  A (continued) 

SQT  2.  MOSC  16J10 

PERFORMANCE  CERTIFICATION  SCORESHEET 
(REFER  TO  INSTRUCTIONS  TO  SUPERVISORS) 


UNIT  OF  ASSIGNMENT 


{ ' 
4 

m 

m 

\ 


1.  At  the  redar  power  supply:  Adjusts  VOLTMETER  to 
Indicate  between  110  and  120  volts. 

2.  Adjusts  LINE  FREQ  neter  to  indicate  between  384  and 
416  CPS. 

(NOTE:  Sequence  of  these  first  two  steps  is  NOT 
important . ) 

3.  At  the  indicator:  Tlirna  POWER  switch  ON. 

4.  At  the  radar  po:wr  supply:  Looks  at  VOLTMETER  and  LINE 
FREQ  meter  and  readjust  if  either  is  outside  the  limits 
specified  in  steps  1 and  2. 

5.  Blower  motors  should  be  heard  operating  in  the  POWER 
SUPPLY,  RECEIVER -TRANSMITTER,  and  the  ANTENNA  BASE  units. 

If  one  of  these  motors  does  not  come  on,  and  examinee 
proceeds,  he  falls  this  step. 

6.  At  the  Indicator:  Looks  at  or  moves  to  and  looks  at 
the  PPI-scope  and  A-scope.  A spot  of  light  should  be 
visible  on  each  scope. 

7.  At  the  receiver-transmitter:  Positions  TEST  SELECTOR 
switch  to  MAG-FIL  and  looks  at  test  meter.  Meter  should 
indicate  between  limits  shown  on  stlck-on  template. 

8.  At  the  indicator:  'Hirns  ANTENNA  ROTATE  control  clock- 
wise and  then  counterclockwise.  If  antenna  Indicator 
fails  to  move,  this  constitutes  an  equipment  malfunction. 

(NOTE:  At  this  point  examinee  must  wait  until  delay 
timer  runs  down.) 

9.  At  the  indicator:  Presses  RADIATE  pushbutton. 

10.  Performs  steps  in  sequence. 

11.  Completes  task  in  10  minutes. 

NOT 

00  NO  GO  OBSERVED 

The  soldier  is  scored  GO  if  he  passes  

all  performance  measures.  | | | | | | 


I I I I 


I I 

iZIi 


I 
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APPENDIX  B 
SQT  NOTICE 

1.  PARAGRAPHS  1-3  will  be  the  seoe  on  all  notices. 

2.  PARAGRAPH  4 will  be  the  written  component. 

a.  Paragraph  4a  Is  standard  except  for  changes  to  the  number  of 
scorable  units  and  MOS  and  skill  level. 

b.  Paragraph  4b  Is  specific  to  MOS  and  skill  level. 

3.  PARAGRAPH  5 will  be  the  hands-on  portion  If  your  test  has  a hands- 
on  portion. 

a.  Paragraph  5a  Is  standard. 

b.  Paragraph  5b  Is  specific  to  the  MOS  and  skill  level.  It  will  be 
transcribed  from  the  Instructions  to  examinee  and  the  score  sheets  In  the 
hands-on  package. 

4.  PARAGRAPH  6 will  be  the  performance  certification  component  If  your 
test  has  a PCC.  If  there  Is  no  hands-on,  the  PCC  will  be  paragraph  5. 


a.  Paragraph  a Is  a description  of  the  purpose  of  the  component  and 
a brief  verbal  description  of  each  task. 

b.  Paragraph  b Is  an  extract  of  the  scoresheet  from  the  performance 
certification  package  with  a standard  Introduction  which  must  be  modified 
to  match  the  skill  level  and  MOS  title. 

5.  HANDS-ON  COMPONENT. 

a.  Ihe  second  part  of  your  SQT  will  be  the  hands-on  component  which 

will  consist  of performance  tests.  Each  test  counts  one  scorable 

unit.  You  will  be  graded  GO  or  NO  GO  by  a scorer  as  you  perform  each 
of  the  tests. 

b.  The  tasks  which  will  be  Included  In  the  hands-on  component  are 

listed  below.  The  task  numbers  and  titles  are  taken  from  the  Soldier's 
Manuals  for  Skill  Levels  and  , (Title  of  MOSC) 


TASK  NUMBER  TASK 
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INSTRUCTIONS  TO  EXAMINEE; 

AT  THIS  STATION  YOU  WILL  BE  TESTED  ON  YOUR  ABILITY  TO  MASK  PROPERLY 
WITH  THE  M17A1  MASK.  WHEN  I GIVE  YOU  THE  ALERT  "GAS,"  YOU  WILL  PUT 
ON  THE  MASK  USING  PROPER  PROCEDURES.  YOU  HAVE  9 SECONDS  TO  COMPLETE 
THIS  TASK.  DO  YOU  UNDERSTAND  WHAT  YOU  ARE  REQUIRED  TO  DO?  "GAS!" 


PASS  FAIL 

1.  Puts  on  and  seats  protective  mask  by  Inserting  chin  . 

flrrt. 

2.  Clears  protective  mask  by  placing  one  hand  over  the  I 

outlet  cover  and  blowing.  I 


3.  Checks  seal  by  placing  hands  over  air  Inlet  valve 
assemblies  and  Inhaling.  Mask  will  collapse. 


K 4.  Completes  all  steps  within  9 seconds  of  the  command 

[I  "GAS." 

j 5.  Perform  all  steps  in  sequence.  , 

I 6.  PERFORMANCE  CERTIFICATION  COMPONENT.  ‘ 

a.  The  purpose  of  the  performance  certification  component  SQT  Is  to  , 

allow  your  commander  or  supervisor  to  certify  your  proficiency  with  the 
M16A1  rifle  and  your  physical  fitness  as  measured  by  the  Advanced  Phys-  ' 

leal  Fitness  Test  (APFT).  You  are  required  to  take  the  APFT  and  qualify 
with  the  M16A1  rifle  trlthln  the  12  months  preceding  the  SQT  quarter. 

Your  performance  will  be  certified  on  the  following  tasks  as  explained 
below. 

b.  The  tasks  which  will  be  included  in  the  performance  certification 

are  listed  below.  The  task  numbers  and  titles  are  taken  from  the  Soldier's  I 

Manuals  for  Skill  Levels  and,  (Title  of  MOSC).  , 


TASK  NUMBER 


SU-(l)  OOO-OOA-2002 


TASK 

QUALIFY  WITH  M161  RIFLE 
(ENGAGE  TARGETS  WITH 
M16A1  RIFLE)  (SPECIFIC 
COURSE  OF  FIRE  TO  BE 
DETERMINE  BY  LOCAL 
CONDITIONS) 


TASK  NUMBER 


SU-(2)  OOO-OOA-2005 


TASK 

TAKE  APFT  (ADVANCED 
PHYSICAL  FITNESS  TEST) 
(MAINTAIN  INDIVIDUAL) 
PHYSICAL  FITNESS) 


ARI  P-77-5 
SCORABLE  UNITS 

Up  to  3 units 

a.  Fall  to  Qualify: 

0 units 

b.  Marksman: 

1 unit 

c.  Sharpshooter: 

2 units 
d«  Expert: 

3 units 


SCORABLE  UNITS 

Up  to  3 units 
a>  Fall  to  Qualify: 

0 Units 

b.  300-399:  1 unit 

c.  400-499:  2 units 

d«  450-500:  3 units 


CHECKLIST  FOR  WRITTEN  SU 
SQT  NOTICE  ENTRY 

1.  Verify  and  ll8.t  the  correct  scorable  unit  number  from  the  test* 

2«  Verify  and  list  the  correct  task  number  from  the  Soldier's  Manual* 

3*  Verify  and  list  the  correct  task  title  from  the  Soldier's  Manual* 

4*  Tell  the  soldier  what  he  will  be  given* 

a*  format  or  problem  (situation,  question) 

b*  media  (graphs,  photos,  tables,  charts) 

c*  tools/aldo  (maps,  protractors,  JPA,  manual  extracts) 

5*  Tell  the  soldier  what  aspects  or  areas  of  the  task  he  will  be  dealing 
with* 

6*  Tell  the  soldier  %diat  specific  products  he  must  select* 

7*  Tell  the  soldier  how  he  will  be  required  to  select  his  answers 
(correct  alternatives,  sequencing,  matching)* 

8*  Tell  the  soldier  how  many  questions  he  will  be  required  to  answer  and 
how  many  he  muat  answer  correctly  for  a 60* 

9*  Show  the  soldier  a sample  item  from  the  SU. 
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SAMPLE 

SKILL  QUALIFICATION  TEST  NOTICE 
FOR 

SQT , MOSC 

1.  THE  SKILL  QUALIFICATION  TEST  (SQT).  The  SQT  is  designed  to  test 
your  ability  to  do  those  tasks  In  your  Military  Occupational  Specialty 
(MOS)  that  are  most  Important  to  your  survival  In  combat,  accomplish- 
ment of  your  Job,  and  your  unit's  mission.  The  SQT  does  not  test  all 
these  Important  tasks  In  any  one  year,  and  the  sample  may  change  each 
year. 


2.  SQT  COMPONENTS.  The  SQT  may  have  as  many  as  three  parts  (components) 
or  as  few  as  one:  Written  Component,  Hands-on  Component,  and  Performance 
Certification  Component.  Each  component  is  made  up  of  Scorable  Units. 

A Scorable  Unit  measures  your  ability  to  perform  a task.  The  tasks 
Included  In  the  SQT  are  taken  from  the  Soldier's  Manual  (SM)  for  the 
MOS  being  tested.  Each  SQT  Is  made  up  of  tasks  taken  from  two  skill 
levels.  Most  of  the  tasks  come  from  the  SM  for  the  skill  level  in 
which  you  are  working  when  you  take  the  SQT.  Some  come  from  the  SM  for 
the  next  higher  skill  level.  The  components  Included  In  this  SQT  are 
described  in  paragraphs  below. 


3.  PREPARING  TO  TAKE  THE  SQT.  In  preparing  to  take  your  SQT,  you 
should  use  your  Soldier's  Manual.  For  each  task  tested  on  your  SQT,  the 
Soldier's  Manual  tells  you  what  you  have  to  do  to  perform  the  task,  the 
conditions  under  which  the  task  Is  to  be  performed,  and  the  standards 
which  the  performance  has  to  meet.  The  task  number  and  title  of  each 
task  on  the  SQT  Is  given  below.  Locate  each  task  In  the  Soldier's 
Manual.  Carefully  read  the  description.  Get  the  TM's,  FM's,  or  other 
materials  referred  to  In  the  Soldier's  Manual  and  carefully  study  the 
sections  which  are  Important.  Be  sure  you  understand  what  Is  required 
to  perform  the  task.  If  you  have  questions,  ask  your  supervisor.  When 
you  clearly  understand  what  the  task  is  and  what  you  arc  to  do  in 
performing  the  task,  practice  the  task.  Get  your  supervisor  or  someone 
else  to  score  your  performance.  Continue  the  practice  until  you  have 
mastered  the  task. 


4.  WRITTEN  COMPONENT.  • 

,a.  Description.  There  will  be  scorable  units  In  the  Written 

Component.  Each  scorable  unit  will  have  one  to  ten  questions.  You  may  , 

be  tested  in  several  diff erect  ways  in  the  Written  Component.  You  may  ) 

have  to  work  through  a problem  and  then  answer’s  series  of  questions  j 

about  what  you  have  done.  You  may  answer  questions  based  on  a situation 
or  drawing.  You  may  have  to  answer  questions  that  show  you  know  how  to  j 

perform  an  action.  All  questions  will  be  based  on  on-the-Job  performance.  | 
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There  will  not  be  any  trick  questions*  The  best  way  to  study  and  prepare 
for  the  SQT  Is  to  practice  the  tasks  according  to  the  Soldier's  Manual* 

b*  Tasks*  The  tasks  which  will  be  Included  In  the  Written  Component 
are  listed  below*  The  task  numbers  and  titles  are  taken  from  the 
Soldier's  Manuals  for  Skill  Levels  and  * (Title  of  MOSC*) 


TASK  NUMBER  TASK 

SU-(l)  OOO-OOA-0000  DETERMINE  FIRST  AID  MEASURES  TO  BE  APPLIED  TO  A 

CASUALTY 

You  will  be  given  two  different  situations  and  will  have  to  answer 
questions  about  each  situation.  The  questions  will  lead  you  through  a 
series  of  events  which  duplicates  the  normal  treatment  sequence  for  each 
type  of  casualty*  You  must  choose  the  correct  treatment  or  treatments 
from  the  possible  answers  provided*  This  unit  will  consist  of  four 
questions*  You  must  answer  all  questions  correctly  to  score  a "GO"  for 
the  unit* 


TASK  NUMBER  TASK 

SU-(2)  OOO-OOA-0001  RECOGNIZE  AND  INTERPRET  MARKERS  OF  AREAS  OF  NBC 

CONTAMINATION  AND  HAZARD 

You  will  be  given  six  drawings  of  markings  with  basic  color  Information 
Identified*  You  must  match  the  correct  shape  and  color  «rlth  the  words 
that  complete  the  marker*  This  unit  will  consist  of  four  questions* 

You  must  answer  all  questions  correctly  to  score  a "GO"  for  the  unit* 


TASK  NUMBER  TASK 

SU-(3)  OOO-OOA-0002  INSPECT  AND  MAINTAIN  THE  PROTECTIVE  MASK  AND  ITS 

CARRIER 

You  will  be  asked  questions  on  Inspection  procedures,  actions  authorized 
for  operator  maintenance,  cleaning,  and  repair  procedures*  You  must 
select  the  correct  action  or  procedure  from  the  alternatives  given.  This 
unit  will  consist  of  five  questions*  You  must  answer  four  questions 
correctly  to  score  a "GO"  for  the  unit* 
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TASK  NUMBER  TASK 

SU-(4)  OOO-OOA-0003  PERFORM  BATTLESIGHT  ZEROING  OF  M16A1  RIFLE 

You  will  be  given  pictures  showing  steps  In  zeroing  the  M16  rifle*  You 
will  have  to  answer  two  types  of  questions*  In  one  type  you  have  to 
Identify  the  problem  shown  In  the  picture*  In  the  other,  you  must  answer 
the  question  by  selecting  the  correct  picture*  This  unit  will  consist  of 
five  questions*  You  must  answer  four  questions  correctly  to  score  a "GO" 
for  the  unit* 

TASK  NUMBER  TASK 

SU-(5)  OOO-OOA-004  DETERMINE  YOUR  LOCATION  ON  A MAP 

You  will  be  given  a map  protractor,  grid  scales,  and  selected  parts  of 
maps*  Your  position  on  the  map  or  the  coordinates  and  the  azimuth  will 
be  given  to  you  In  the  question*  You  must  complete  the  necessary  calcu- 
lations or  conversions  asked  for  and  select  the  correct  answer*  This 
unit  will  consist  of  four  questions*  You  must  answer  four  questions 
correctly  to  score  a "GO"  for  the  unit* 

TASK  NUMBER  TASK 

SU-(6)  OOO-OOA-0009  PERFORM  EQUIPMENT  SERVICEABILITY  CRITERIA 

(ESC)  INSPECTION* 

You  will  be  given  a portion  of  an  Equipment  Serviceability  Criteria  (ESC) 
and  asked  questions  about  the  performance  of  an  ESC  Inspection*  You  will 
be  given  also  a symptom  In  the  question  and  be  required  to  Identify  the 
correct  status  to  place  on  the  part  of  the  equipment  that  has  the  symptom* 
You  must  select  the  correct  answer  from  those  given*  This  unit  will 
consist  of  three  questions*  You  must  answer  three  questions  correctly  to 
score  a "GO"  for  the  unit* 


TASK  NUMBER  TASK 

SU-(7)  OO-OOA-011  PREPARE  TAMMS  FORMS  USED  BY  VEHICLE  OPERATOR 

You  will  be  given  a partially  completed  DA  Form  2400  (Equipment  Utili- 
zation Record)  and  a blank  DA  Form  2404  (Equipment  Inspection/Maintenance 
Worksheet)*  In  the  situation,  you  will  be  assigned  as  a driver  of  a vehi- 
cle and  asked  to  Identify  where  entries  are  made  on  the  DA  Form  2400 
and  2404  or  asked  questions  about  the  use  of  the  forms*  Yo  must  choose 
the  correct  answers  from  sets  of  possible  answers*  This  unit  will 
consist  of  three  questions*  You  must  answer  three  questions  correctly 
to  score  a "GO"  for  the  unit* 
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GO  NO  GO 


UNIT  1 


TASK  NUMBER 


54-2  OO-OOA-1002 


INSTALL  AND  OPERATE  FIELD  TELEPHONE 
TA-312/PT 


INSTRUCTIONS  TO  THE  EXAMINEE:  YOU  WILL  INSTALL  AND  OPERATE  A TA-312/PT, 
FIELD  TELEPHONE.  THE  FIELD  WIRE  YOU  SEE  HERE  IS  CONNECTED  TO  ANOTHER 
FIELD  TELEPHONE  WHICH  WILL  ACT  AS  A SWITCHBOARD  FOR  YOU.  ALL  EQUIPMENT 
NEEDED  TO  INSTALL  AND  OPEPJ^TE  THE  TA-312/PT  IS  PRESENT.  I CANNOT  HELP 
YOU  DURING  THIS  TEST.  YOU  HAVE  FIVE  MINUTES  TO  COMPLETE  THIS  TASK.  DO 
YOU  UNDERSTAND  WHAT  YOU  ARE  REQUIRED  fO  DO?  YOU  MAY  BEGIN. 

PASS  FAIL 

1.  Uses  pliers  TL-13A  to  strip  insulation  from  each 

conductor  of  the  wire  line.  

2.  Presses  spring-loaded  line  binding  posts  and  Inserts 

one  conductor  Into  each  post.  

3.  Adjusts  buzzer  volume  control  knob  to  LOUD  position.  

4.  Places  INT-EXT  switch  to  INT.  

5.  Places  CB/LB/CBS  switch  to  LB.  

6.  Inserts  BA-30  batteries  Into  the  battery  compartment, 

one  up  and  one  down.  

"NOW  RING  THE  OPERATOR  ON  THE  OTHER  END  AND  ASK  HIM  TO  RING 
YOU  BACK." 

7.  Turns  handcrank  rapidly  at  least  two  turns.  

8.  Removes  handset  from  retaining  cradle  and  waits  for 

operator  to  answer.  

9.  Presses  PRESS-TO-TALK  switch  and  asks  operator  on  other 

end  to  ring  back.  

10.  When  buzzer  rings,  terminates  call  and  places  handset  In 

retaining  cradle.  

11.  Performs  first  six  steps  before  step  seven.  

12.  Completes  task  in  five  minutes.  


UNIT  2 


GO  NO  GO 


*1 


